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CHARTER I 


THE VALUE AND USES OF TRADE TESTS 


Vocational schools, as opposed to schools for general education, 
have as their avowed aim the preparation of youth to earn a living 
in vocations. 

The extent to which their aim is achieved by such schools as are 
called vocational schools is at present a debatable issue. One reason 
for this is the lack of means with which to measure the human 
products of such schools. Inability to know the product of the 
school at once predetermines an inability to better the school 
administration on a basis of the needs of students. Any betterment 
in present methods of measuring the product of a school at once 
_ produces the chance for bettering the school administration 
on the basis of the needs of the students. Only until very re- 
cently has trade skill and knowledge been measured in more 
refined terms than subjective personal judgments of “skilled man” 
and “helper,” “successful” and “unsuccessful,” “competent” and 
“incompetent.” Army trade tests solved this problem for the army; 
adaptations of the method may solve the problem for our vocational 
schools. Graduates of our vocational schools are at present of 
equal merit, so far as measure of their merit goes, for they all re- 
ceive diplomas. Industry wants a more accurate measure of their 
hiring worth; and the school desires to recognize the varying merit 
of its product. 

Unless the prospective employer can successfully rate the human 
product of the vocational school he is at a loss to know in placing 
a graduate of a vocational schoo] in industry whether he should 
adopt a coursedifferent from that followed in the caseof any other ap- | 
plicant for the job. The average employer is skeptical of the value of 
the “book learned” tradesman. His skepticism, we must believe, 
is not directed so much toward vocational school graduates as 
against particular exceptional individuals in the past who have 
happened to fail when given the test of industry, the test of doing 
the job itself. Could graduating students be rated for general trade 
proficiency against men in the trade itself, much would be gained in 
the way of proving just what is the value of any specific vocational 
training program. 
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The percentage of a graduating vocational school group which 
equals or exceeds in general trade skill the median tradesman in 
industry is a theoretical measure of the efficiency of the vocational 
school instruction. The army trade tests, standardized on random 
sampling of men in industry, fulfill these requirements of measures 
of final trade proficiency after a period of training. They may be 
used equally well to measure trade proficiency at any point of its 
acquirement. ? 

Vocational school instruction has so generally meant trade 
school instruction that the author will not attempt here to make a 
distinction between the two. It is evident that there is no dis- 
tinction between the two in the field of tests, which have a common 
statistical technique and hence make no distinction, save in content, 
between intelligence tests, educational tests, vocational tests, trade 
tests, or any other variety of test which has as its aim the pre- 
diction of peoples’ abilities to do something, or to act in a certain 
way. Accordingly, the terms “trade test” and “vocational test” will 
be used interchangeably. 


VALUE OF TRADE TESTS IN EDUCATIONAL ADMINISTRATION 


The values of trade tests in education are such as pertain to 
any objective, standardized trade examination method. Standard- 
ized tests, or tests designed by taking into account the testing princi- 
ples thus far discovered, will have a better chance of fulfilling 
the demands made upon them than tests not so constructed. 

For argument’s sake, one may take for granted the assumption 
that a vocational education course under consideration is a desir- 
able course, and that it is as good a course as may be expected to 
be produced. Not all students will apply themselves equally to the 
tasks of learning the trade instructions given them in such a course, 
nor profit equally from those received. Tests of progress, given at 
periodical intervals, are needed to keep pupils up to a reasonable 
standard of interest in their courses and for purposes of grading 
students at the end of the year, or at the end of the course. If such 
tests can be standardized, then so much the better. But if not, as 
will often be the case, then vocational instructors may be able to 
profit greatly by attention to the methods of question making and 
test formulation used in army procedure. 
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Of two students of equal general trade or vocational ability, \ 
one may be unusually good in operation A and be unusually poor in 
B; the other unusually poor in A but good in B. If, during the 
course of instruction, the strengths and weaknesses of each student 
could be made known, much valuable time of the student might be 
saved, and a much better allround ability gained by the teacher’s 
adapting his teaching to individual needs. Diagnostic tests of an- 
alyzed or subdivided elements of general trade ability are needed. 
These will be developed in the future. A knowledge of what the 
army was able to do in measuring trade ability should help the 
prospective test maker in making up such tests. Where such tests 
have been developed to the point where they are sufficiently accurate 
for the purpose, we may expect trade instruction to be done largely 
by the job or operation sheet method, and the pupil to be advanced 
from job to job or operation to operation just as fast as he shows a test 
measured proficiency of X in each job or operation. Thus will result 
economy of time required in learning a trade, since each pupil will 
progress just as fast as he is capable of doing. This is the method 
found successful by the S.A.T.C. during the war, and since 
adopted by the E. and R. Schools of the Army in teaching trades to 
thousands of boys. In the past the test of whether or not a given 
pupil was ready for promotion to a new job has been a subjective 
estimate by the instructor, aided by performance on the job, it is 
true, but nevertheless a test which could not be duplicated by an 
outside person without a personal knowledge of the pupils being 
rated. 

Tests of promise or capacity have been used in the field of general 
education for a number of years; they have not been so generally 
used in trade education. Much research work, greatly encouraged 
by the success of the army intelligence tests, has demonstrated the 
feasibility of classifying school children on the basis of their mental 
ability, or on the basis of a composite score of a number of tests 
designed to predict pupils’ abilities to profit by the instruction 
given them. The E. and R. Schools of the Army have thus classified 
many thousands of soldiers, drawn from every part of the country 
and from foreign countries, with varied and non-comparable pre- 
vious school experiences. The value of such educational guidance 
has been already proved. 

Trade test methods, making use of refined tests of trade capacity, 
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promise to throw some light upon the age-old problem of “What 
vocation should a boy enter?” Whether a -boy should take up 
general education, trade education, or professional education, is a 
problem which we may hope to solve only by development of a 
vocational philosophy formulated with the aid of pertinent facts 
of human abilities and vocational opportunities made available by 
the statistical , methods common to all survey methods. In con- 
_junction with ‘psychological tests> 2 physical tests and “questionnaires, 

| () tests of trade capacity may be expected to become a valuable aid 
to the vocational counsellor of the future. 

The best large vocational schools have their,own placement or em- 
ployment bureaus. The problem of such bureaus is to evaluate into 
one single fitness score, by a subjective process, the diversified 
talents of boys and girls as a basis for a twofold categorical divi- 
sion of resulting action into “recommended” and “not recommended” 
for the prospective position, whether job in industry, or school 
instruction in an advanced school. Such estimates, when based 
upon proficiency in performing trade operations, may be greatly 
enhanced by more accurate methods of measuring trade skill. The 

* two aspects of such placement are: (a) knowledge of the pupil’s 
abilities, and (6) the requirements of the job, in the way of human 
abilities and acquirements./ Text-books on vocational guidance give 

\}many facts about trades, tradesmen and trade conditions. They 
\ generally fail to specify how diverse human talents for the job, or 
diverse demands of the job upon the worker, may be evaluated. 
These problems will be ultimately solved by methods available to, 
or to be developed by, the test statistician. 

This larger aspect of “fitting the job and the man” is even more 
important than the more restricted problem of measuring the 
talents for a job, present or future, measurable by trade tests. All 
make use of a common statistical technique. Accordingly, we 
should mean by “trade test methods” this larger aspect of evaluat- 
ing measurable human talents in terms of some independent 
criterion of job proficiency. Inasmuch as tests of present trade 
proficiency are promises of future performances (immediate or 
remote) on the job, all trade tests in the last analysis are tests of 
promise or capacity. The common statistical problem of all such 
methods is the problem of measuring mutual dependence of related 
measures of trade capacity. 
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CLASSIFICATION OF TRADE TESTS AND EXAMPLES OF TYPICAL FORMS 


The term “trade test” is used to refer to any test involving test 

aterial taken from the trades. )Trade tests differ, then, from any 
other forms of tests mainly in a difference in content. They may be 
categorically divided into two classes according to the purpose of 
the test, and further sub-classified according to form of test material] 
and method of administration, as follows: 


I. Tests of proficiency or success in a trade. 
1. Verbal. 
a. Oral Administration. . 
(1) Army Oral. 
\(a) Oral Answers. 
(6) Written Answers. -- 
»(2) Picture. — 
bs-Written Administration. 
~ (1) One-word-answer. 
“<(2) Multiple Choice. 
2. Performance Tests. 


Il. Tests of trade capacity, potentiality, or promise. 


Sub-classes of this group: Practically every known variety of 
test, including the above verbal forms, has been used by 
some investigator at one time or another. 


Trade tests were used in the army to determine which men, of 
those claiming specific trade experience, were sufficiently competent 
to be detailed for special duty in the trades maintained in the army. 

An example of an army oral form of test, used for measuring trade 
proficiency, is reproduced herewith in an eyelet machine operator’s 
test. This test may be given individually and orally by an examiner, 
the answers being oral; it may be given orally to a group, the 
answers being written on blank paper; or it may be administered 
by the use of printed or mimeographed questions, the answers being 
in the one-word-answer form. 


EYELET MACHINE OPERATOR (TENDER) 


(Army oral form of test with answer in the one-word-answer form.) 
1. What do you call the large main shaft at the top of the machine? 
Ans. Cam. 
2. What rubs on the cams al! the time to oil them? 
Ans. Tassel. 


Io. 


21. 


22. 


. 
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What does the scrap wind up on? 
Ans. Reel. 


. What drives the reel from the bottom cam shaft? 


Ans. Belt. 


. What do you call the die which does the first operation? 


Ans. Blanking. 


. What is the name of the large part which moves back and forth and carries 


the work from one die to another? 
Ans. Slide. 


. What do you call the part which moves up and down and carries the punch 


at its lower end? 
Ans. Plunger. 


. What do you call the pulley on the drive shaft which is not fastened to the 


shaft? 
Ans. Loose. 


. What is the shaft underneath the machine called? 


Ans. Cam. 

What are the small springlike parts called which grab the work and carry it 
from one die to another? 

Ans. Finger. 


. What do you use to measure the thickness of the metal? 


Ans. Micrometer. 


. What do you use to cut off the strip metal? 


Ans. Shears (snips). 


. What is used to tell if the finished work is the correct size? 


Ans. Gage. 

What do you call a gage which shows the largest and smallest size that can be 
allowed on a finished piece of work? 

Ans. Limit (maximum and minimum) (“max and min”). 


. Of what metal are the reels made? 


Ans. Brass. 

What is the shape of the plungers? 

Ans. Square. 

When five rows of blanks are to be punched from the strip metal, which 
row is punched first? 

Ans. Middle (center). 

What is there on the end of the air pipe to let the air out at just the right time? 

Ans. Valve. 

How many holes are there in the drive pulley to turn it by hand? 

Ans. 2 (4). 


. What do you call the tool used to turn the fly wheel by hand? 


Ans. Bar. 
What part of the cam shaft regulates the air? 
Ans. Cam, 


If the top cam shaft turns at 60 revolutions per minute how many revolu- 
tions per minute does the bottom cam shaft turn? 
Ans. 60 (same). 
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23. What guides the finished work into the pan? 
Ans. Spout (pipe). 

24. What is the sheet iron around the gears called? 
Ans. Guard. 

25. What does the pan for the work set on? 


Ans. Box. 
26. What do the dies set in? 
Ans. Holder. 


27. What tool do you use to turn the roll feed? 
Ans. Wrench, 
28. What material is used to line a brake on machines which have a brake on the 
drive shaft? 
Ans. Leather (wood). 
29. How is the punch fastened to the plunger? 
Ans. Screws in (wrench) (thread). 
30. What is the washer made of that is sometimes used on the reel shaft? 
Ans. Leather. 


31. What do you call the attachment sometimes used around a punch to make 
the work let loose from the punch? 

Ans. Thimble. 

32. How are the bottom cams made so that they can be timed? 
Ans. Split (halves) (screw). 

33. What too] do you use to set the bottom cams on the bottom cam shaft? 
Ans. Screw-driver. 

34. Of what material is the part of the slide friction made that rubs on the slide? 
Ans. Wood (maple). 

Army picture tests were administered orally and the answers 
- given orally. They were used as a check on the ratings made on the 
oral tests, which are somewhat subject to language difficulties and 
to coaching. The method is essentially the army oral method, but 
with pictures used to describe the trade situation in the place of the 
descriptive phrases or clauses which must be used in the oral method. 
The lathe hand test, herewith reproduced, pages 8 to 14, is typical 
of this form of examination. 

The performance test aims to duplicate a shop situation, the test 
subject being rated on the quantity and quality of work done on a 
job or problem assigned. These have the merit, not possessed by 
ordinary shop jobs, of being so chosen as to require a maximum 
amount of manipulation of tools, use of trade knowledge and trade 
skill. A sample performance test, used in testing ability to use 
machinists’ measuring tools, is shown on page 15. This particular 
test has as its products a series of dimensions entered on a form 
blank, which are scorable by stencil. 
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Tests of trade proficiency were developed.later than tests of trade 
capacity. Tests of trade proficiency were developed to a high point 
of efficiency during the War. The possibilities of large scale re- 
search in developing tests of trade capacity have not yet been 
tested. It is the special function of this research to point out the 
value of such work by statistical inquiry into particular selected 
vocations and trade school methods. 


LATHE HAND TEST 
Ricruriiner 


1. What is the part at G called? 
Ans. Steady-rest. 
2. What letter shows the live center? 
Alas 1D). 
3. What is the lever at K used for? 
Ans. Screws (threads). 
4. What do you call the part at C? 
Ans. Face-plate. _ 
5. What do you call the lever at A? 
Ans. Back-gear. 
6. What is the part at L used for? 
Ans. Gears (change the gears). 
7. What letter shows the cone? 
Ans. B. 
8. What letter marks the tail stock? 
Ans.) ll 
g. What is the part at F called? 
Ans. Follower-rest (follower). 
10. What is the part at J called? 
Ans. Rack. 
11. What is the part at I called? 
Ans. Lead-screw. 
12. What is the part at E called? 
Ans. Tool-post. 


PICTURE 2 

13. What does that micrometer read? 
Ans, .125,. 

14. What is the part at A called? 
Ans. Ratchet. 


PICTURE 3 

15. What do you call! that lathe tool? 
Ans. Dog. 

16. What does the part at A fit into on the lathe? 
Ans. Face-plate (slot on face-plate). 


| ~F1g.8 


PLATE I. PAGE 1 OF LATHE HAND PicTuRE TRADE TEst 
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PICTURE 4 

17- What do you call that tool? ‘ 
Ans. Boring. 

PICTURE 5 

18. What is that tool called? 
Ans. Dividers. 

PICTURE 6 

19. What do you call that lathe tool? 


Ans. Center. 
20. How many degrees in the angle of the point of that center? 


Ans. 60. 
PICTURE 7 
21. What do you call that tool? 
Ans, Center-punch. 
PICTURE 8 
22. What is that tool used for? 
Ans. Threads (screws). 
PICTURE 9 
23. In what operation is that tool used? 
Ans. Drilling. 
PICTURE IO 
24. What letter marks the apron of that lathe? 


Ans. G. 
25. What letter marks the bed of that lathe? 
Ans. H. 
26. What letter shows the part which you turn to throw the longitudinal feed in 
gear? 
Aas, (Cc 
27. What letter shows the part which you turn to throw the automatic cross-feed 
in gear? 
Ans. WD. 
28. What letter marks the part which works the compound-feed? 
AOS: Ie 
29. What letter shows the swing of that lathe? 
Ans. A. 


PICTURE II 

30. What operation is being done in that picture? 
Ans. Boring. 

31. What is the part at A called? 
Ans. Face-plate. 

32. What is the part at B called? 
Ans. Dog. 


F1g: Wile 
Flg./8. 
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33. What is the part at C called? 
Ans. Boring-bar. ~ 
PICTURE 12 
34. If the lead screw on that lathe has 8-threads per inch, how many threads per 
inch will that lathe cut with those gears? 
Ans. 16; 
35. What is the gear at A called? 
Ans. Idler (intermediate). 
PICTURE 13 
36. What is the tool in that picture called? 
Ans. Morphadite (hermaphrodite). 
PICTURE 14 
37. What tool is being squared-up in that picture? 
Ans. Threading. 
PICTURE 15 
38. What operation is being done in that picture? 
Ans. Taper. 
39. If the stock in that lathe is 2-feet long and you want a taper of 14-inch per 
foot, how far do you set-over the screw G? 
Ans. % (%-inch). 
PICTURE 16 
40. What tool is being squared-up in that picture? 
Ans. Inside-threading. 
PICTURE 17 
41. What do you call that tool? 
Ans. Vernier-caliper. 
42. What is the smallest fraction of an inch which that will measure? 
Ans. 1/1000. 
PICTURE 18 
43. What do you call the part at B? 
Ans. Collet (collar) (washer). 
44. What do you call a slot like that at A? 
Ans. Y-slot. 
45. How far does one revolution of the handle at C advance the point of the 
thread tool? 
Ans. % (%-inch) (.125). 
PICTURE 19 
46. What is the name of the tool at 1? 
Ans. Left-hand-facing. 
47. What is the name of the tool at 2? 
Ans. Right-hand-facing. 
48. What is the name of the tool at 4? 
Ans. Diamond-point. 
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49. What is the name of the tool at 6? 
Ans. Round-nose. . 
50. What is the name of the tool at 7? 
Ans. Parting (cutting-off). 
51. What is the name of the tool at 9? 
Ans. Bent-threading. 
52. What is the name of the tool at 12? 
Ans. Inside-threading. 


PICTURE 20 
53. What gear do you use on the lead-screw to cut a screw of 13 threads per inch, 
if you have a 24-tooth gear on the spindle? 
Ans. 39. 
54. How many threads per inch on a 1-inch pipe thread? 
Ans. 11%. 
PICTURE 21 


55. What kind of work has been done on that stock? 
Ans. Wnurling. 

PICTURE 22 

56. What do you call that lathe tool? 
Ans, Drill-chuck. 


57. What do you call the set-screws on that chuck? 
Ans. Safety. 


PICTURE 23 

58. What do you call the part at B? 
Ams. Chuck. 

59. What do you call the part at A? 
Ans. Turrett. 


PICTURE 24 

60. What is being turned-up in that picture? 
Ans. Crank-shaft. 

61. What is the part at A called? 
Ans. Jig. 

62. What do you use to prevent vibration when turning off-center work? 
Ans. Weight (balance) (counter-balance). 


PICTURE 25 


63. What do you call that tool? 
Ans. V-block. 

64. What is that tool used for? 
Ans, Centering? (laying-out). 


PICTURE 26 


65. What do you call that tool? 
Ans. Depth-gage. 
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Tests of trade capacity, potentiality, or promise range widely in 
content, administration, and test method. At one end of the range 
of tests so used in the past, we have the simple tests of psycho- 
physical measurements; at the opposite end, we have lengthy 
questionnaires aiming at measuring amateur ability in the trade, 
and initial demonstration trade lessons with performance tests 
thereon, designed to measure ability to learn simple processes of 
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the trade. The apparatus used in administering such tests has varied 
from a piece of paper and pencil to elaborate apparatus designed to 
imitate the motions of the job while automatically recording 
quality and quantity of performance. Such tests have been used to 
predict probable wages, amount of production per hour, foreman’s 
rankings of workers, accident risk, school marks, chance of being hired 
when referred to a job, probable length of stay on the job, and so on. 

As typical of one of the most recent developments of tests of trade 
capacity might be mentioned a pasting test used at the Manhattan 
Trade School for Girls) New York City. The teacher, with all 
required materials before her, demonstrates the making of a paper 
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covered pasteboard box. Each step in the operation is explained: 
the use of tools, position of fingers, position of materials, chances 
for doing poor work which must be avoided by the pupil. At the ° 
completion of the demonstration each child takes her seat, and 
is provided with paste-pot, paste, brush, wipe-rag, scissors, past- _ 
ingboard, pasteboard, colored paper, cloth for reénforcement of 
covers, and rule. Without further instruction the pupil makes the 
best box she can, guided by the instructions remembered. The final 
score on the performance is subjectively judged by the teacher for 
“accuracy, neatness, speed and handling,” which four are finally 
subjectively evaluated into an “estimate of pasting ability.” It will 
thus be seen that the test involves ability to imitate recalled di- 
rections and demonstrated trade performance; in short, ability to 
learn a new trade job. 

It is unnecessary to emphasize the fact that, in so far as test 
technique is concerned, there is nothing essentially new or different 


, in a trade test from tests used in the past, called intelligence tests, 
| educational tests, or vocational tests. The growing use of the 
_ statistical method of partial correlation has helped to emphasize the 


fact that any measurable variable fact about a group of people 


, may be used to predict any other variable fact about that same 


group with which it correlates either positively or negatively. 


Almost any verbal intelligence test will be found to be a fair predict- 
er of school marks; school marks may, in certain vocations, be fair 
predicters of probable after-school success in industry; marital 
condition may be a fair predicter of a man’s chances of being hired 
when referred to a prospective employer for a job. The essential 
common basis of all such scientifically constructed tests,—and trade 
tests in particular,—are: (a) determination of a numerical value for 
both variables, test score and measure of success, on a group of 
persons of known or judged ability, and (b) determination of the 
correspondence of test score to job ability to be predicted, on the 
hypothesis that the most probable ability of a second person, of un- 
known ability on the job, but with a given test score X, is the 
ability of the standardization group of persons which corresponds 
to the test score X. The variable, degree of success on the job, 
which is to be predicted, is generally known as the “criterion.” Due 
to lack of standardization of workmen’s job status in industry, 
a criterion of trade ability is unusually hard to establish. The 
statistical technique used is employed in other kinds of tests as well. 


GHAP TER fl 


TESTS OF TRADE PROFICIENCY OR SUCCESS, AND 
THEIR ADAPTATION TO SCHOOL WORK 


COMPARISON OF THE TESTS OF THE INTERVIEW AND “HIRING ON 
TRIAL” WITH THE ORAL TRADE TEST METHOD 


The interview has long been the basis upon which men have been 
judged in regard to their fitness for jobs. When questions, specifi- 
cally applying to the qualifications for the job, have been asked the 
applicant, these have formed one kind of trade test. 

Previous to the war, little constructive thought had been given 
to the methods of examining tradesmen. Many studies of labor 
turnover had clearly indicated that the causes inherent in the way 
men are fitted into jobs are often the major causes of such labor 
turnover. Ways of bettering the selection or examining process 
were slow in being developed. 

One of the most common methods of selection of men has been 
the process of “hiring on trial.” The costliness of the method in 
time spent in supervision of the new employee, spoiled materials, 
broken machinery and dissatisfaction of the employee judged in- 
competent after trial make the method too costly for present-day 
use. Consequently, one may say that present-day general practice 
is to hire men by an interview given them at the employment office. 

Some of the most apparent defects in the method will be discussed, 
for it is just such inferior methods of examination that any gradu- 
ate of any school has to face when applying for a job. 

1. The applicant is seldom given many pertinent questions 
about the job for which he is being examined. The routine of filling 
out a qualification card usually takes so much time that little time 
is given to the much more important consideration of the trade qual- 
ifications possessed by the applicant. Statistically, the objection 
is that where few questions are used, the ratio of the P.F. of an in- 
dividual score to the magnitude of the score is too large. 

2. Not all applicants get the same set of questions, but each 
applicant receives the questions which come to the examiner’s mind 
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on the spur of the moment. Even if the evaluation of the questions 
were objective, comparable ratings would be impossible. Depending 
upon the questions asked, the difficulty of examinations received by 
successive applicants varies greatly. 

3. Because of insufficient care in wording the questions, the 
questions which are asked may be generally answered satisfactorily 
by a person with only a very superficial knowledge of the trade 
process referred to. Thus questions are often merely a test of the 
workman’s ability to understand the language of the examiner 
rather than a test of his trade knowledge or skill. 

4. “Trick” questions have been common. The assumption un- 
derlying these seems to be a feeling that the truly expert tradesman 
will not be caught by the subtle trap set for him by the examiner. 

5. Guess questions, or questions adequately answerable by 
“yes” or: “no,” have often been used. This form of test, commonly 
known as the “true-false” test, is a common form of psychological 
test method. In order to avoid a high P.E. of the individual score, 
test makers generally use a large number of such questions and em- 
ploy a scoring formula which penalizes errors. 

6. Owing undoubtedly to the seeming failures of the verbal 
method in the past, a very general belief has arisen among interview- 
ers, and trades people in general, that the only reliable way to test 
a workingman’s trade ability is to watch the workman at his work. 
Consequently, the trade examination is often slighted. Practically 
no attempts, outside of a few experiments by psychologists, have 
been made to prove statistically the worth of one examination 
as against any other. 

7. Using the common type of question demanding long ex- 
planatory answers, much time would necessarily have to be devoted 
to the trade examination in order to sample but a very few of the 
elements of trade skill and knowledge possessed by the tradesman. 

8. Owing to the time limitations and necessity of using as ex- 
aminer a person well versed in the trade and in methods of oral 
examination, it is impossible to use the method where great numbers 
of persons in various trades might be desired to be examined with 
many pertinent trade questions. One of the greatest advantages of 
the personal oral examination is the use of pertinent follow-up ques- 
tions, used by the examiner as a sort of trade cross-examination 
method. The method is not adaptable to written trade examinations, 
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such as are desirable, and often necessary, in all trade or vocational 
school examinations. 

9g. In any follow-up of graduates of a trade school, for instance, 
the only available statistical record kept of the oral interview is the 
note on the qualification card of “hired” or “not hired.” Such cate- 
gorical ratings are entirely too crude for accurate statistical studies. 
Furthermore, the judgment thus given is a composite subjective 
judgment of not only the results of the trade questions but of the 
qualification card entries as well. It is thus impossible to tell how 
much weight the one factor may have had, as against the others, in 
determining the final categorical rating. 

10. For any reasonable approach to accuracy, the scoring of the 
answers given demands an expert in the trade to evaluate the trades- 
man’s answers. In any modern large industrial employment office, 
men are hired in many trades and for, literally, hundreds of different 
jobs. It is obviously impossible for one man to be intimately ac- 
quainted with the trade processes of so many trades and jobs. The 
self-styled “expert” examiner, after having once discovered a few 
trade questions on each trade, usually lets his examinations there- 
after be wholly made up of the few trade questions which come to 
mind on the spur of the moment. The questions are easily coached 
for by intelligent workmen, while the method fails utterly as do all 
oral methods, in the case of illiterate or foreign-born workmen. 

Many of the above enumerated disadvantages of the present 
verbal methods of examination are overcome by the army oral ex- 
amination. This examination owes its value largely to the methods 
used in obtaining the information which will make good trade ques- 
tions, and in the methods of wording the questions. These advan- 
tages are as follows: 

1. The questions are assembled in conference with experts in the 
trade, only those questions being selected for standardization which 
have been proved to be answerable by a majority of the expert 
tradesmen in the trade in question. The questions are thus, first of 
all in the assembly process, selected questions; they are not such 
questions as expert opinion might be convinced were desirable 
questions, but rather questions such as tryouts on experts in the 
trade have demonstrated to be pertinent questions. 

2. The questions are later standardized on men in industry of 
known different degrees of ability, and a further selection of ques- 


20 Trade Tests in Education 


tions is made upon the statistical basis of proved differentiation of 
different degrees of trade skill on the part of each question used in 
the final test. 

3. Due both to the clearness in meaning to the applicant of the 
questions and to the short key-word answers demanded by the 
questions, many questions may be given in a short time. When 
examining expert tradesmen, a speed of as many as three or even 
four questions given and answered per minute is possible. This 
means that many samplings of diverse elements of trade ability may 
be made in the same time as formerly devoted to but few questions. 

4. The key-word scoring method makes the scoring thoroughly 
objective. All possible allowable answers are predetermined at the 
time of the compilation of the examination. Consequently, any per- 
son competent to recognize the printed answer is practically as 
competent an examiner using the questions as any expert in the 
trade, and more competent than a trade expert without the 
questions. 


THE TECHNIQUE OF TRADE TEST CONSTRUCTION 
ORAL TRADE TESTS 


In order to formulate good oral trade test questions of the one- 
word-answer type one need but remember the simple rule: Decide 
first what is to be the answer to the question, and then, by adminis- 
tration of the question to tradesman, progressively revise the form 
of the question until there are no other allowable correct answers 
than the expected one (or synonomous or alternative answers). 

The gain in objectiveness of administration and scoring made 
possible by the one-word-answer oral question may be shown by the 
two following examples of army oral trade tests on the same trade, 
issued almost a year apart. The first, or earlier one, has the advan- 
tage over ordinary tests used in industry of being compiled from 
questions which have been given a standardization tryout on work- 
men in the trade. The second, or later one, has all the additional 
advantages of almost perfect objectivity of scoring. 


1. ARMY ORAL TRADE TEST—PATTERN-MAKER, WOOD 
Issued April 17, 1918 


1. Why is draft put on patterns? 
Ans. Toenable the molder or foundryman to draw the pattern cleanly from 
the mold (sand). Score 4. 


10. 


Adaptation to School Work 21 


What is meant by rechucking a piece of wood in the lathe? 
Ans. (1) When a turned piece cannot be completed in one operation, it is 
removed from the lathe and set up again in such a position that 
other parts, or the reverse side of the piece can be turned. Score 4, 
(2) Reversing and turning the finished side toward the face plate, 


Score 4, 

. What is the least draft per foot in good molding practice? 
Ans. (1) One-sixteenth to one-eighth of an inch per foot. Score 4. 
(2) One-thirty-second of an inch per foot. Score 2. 


What are two advantages of skeleton patterns? 
Ans. (1) a. Save labor. 
b. Save time. 
c. Save material. (Any two) Score 4 


When gluing up stock for a pattern, which sides of the boards, as related to 


their position in the tree, should be placed together to keep the 
pattern from warping? 


Ans, The like sides (two heart sides) (two outsides). Score 4. 
Explain how to glue the end grain of wood. 
Ans. (1) Clean the grain, size with hot glue, and when dry coat with hot glue; 


place the pieces together and clamp. Score 4. 
(2) Size it with glue and then glue it again. Rub together; glue and 
clamp. Score 4, 


. State two ways provided by the patternmakers for making cores. 


Ans. (1) a. By core boxes (solid boxes). 
b. By sweeps (strickles). (Both required). Score 4. 
In making a pattern from which a brass working pattern is to be made, how 
much shrinkage should be allowed to insure the proper size of 
gray iron castings to be made from the brass pattern? 
Ans. (1) First allow 3/16 inch to the foot to take care of shrinkage of brass 
pattern, and then 1/8 inch to the foot to take care of shrinkage 
on gray iron castings off brass pattern; total of 5/16 inch allow- 


ance to the foot. Score 4. 
(2) 5/16 inch. Score 4. 
(3) 3/16 inch. Score o. 


How much shrinkage in the diameter would there be to an aluminum cast- 
ing two feet in diameter and one-half inch thick? 

Ans. Three-eights to five-eighths of an inch. Score 4. 

State how to find the mitering angle for any number of arms in a pattern 
for a wheel. 

Ans. (1) By dividing 360 (the number of degrees in a circle) by the number 
of arms. (NorTE: specific example is a sufficient answer). 


Score 4. 
(2) Divide the periphery or circumference of the pattern into as many 
equal parts as there are arms on the wheel. Score 4. 


(3) With trams (dividers). Score 2. 
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11. In making a solid cylinder pattern, on which side would you put the tapered 


core print? 
Ans. (1) Cope side. Score 4. 
(2) Drag side. Score 0. 


RATING THE CANDIDATE 


Rate N- men scoring less than 15. 
Rate A— men scoring 16 and 17. 
Rate A men scoring 18 to 31. 
Rate A-++ men scoring 32 and 33. 
Rate J— men scoring 34 and 35. 
Rate J men scoring 36 to 39. 
Rate J+ men scoring 40 and 41. 
Rate E men scoring 42 or more. 
There will be no E— or E+ rating. 


2. U. S. EMPLOYMENT SERVICE TECHNICAL INTERVIEW. 
PATTERN-MAKER, WOOD 


Issued February 2, 1919 


1. What wood, besides pine, is most commonly used for making small patterns? 
Ans, Mahogany. 
2. With what is the surface of a pattern coated to keep it from getting damp 
and warping? 
Ans. Shellac. 
3. How is a pattern made so that it can be drawn out of the sand easily? 
Ans. Draft. 
4. What is the box called in which the sand for a mold is rammed up? 
Ans. Flask. 
5. For what is the allowance made on a pattern-maker’s rule? 
Ans. Shrinkage (contraction). 
6. What do you call the part of the pattern which is above the parting line ona. 
two-part flask? 
Ans. Cope. 
7. What tool do you use to lay out a pattern with a 3-foot radius? 
Ans. Trammels. 
8. In making a large ring pattern, what are the separate parts called? 
Ans. Segments. 
g. What do you call a mold that has a metal face to harden the casting? 
Ans. Chill. 
10. What do you put on a pattern to support the core in the proper position? 
Ans. Prints. 


EXAMINER: Ask at least six questions; more, if necessary, to accurately judge 
the candidate’s ability. Score only the results of the six questions you ask first. 
Vary as much as possible your selection of the first six questions. 
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ASSEMBLY AND REVISION OF ORAL TRADE QUESTIONS AND 
PICTURE TRADE TEST QUESTIONS 


Before assembling any questions from the experts in the field, one 
should first determine just what the trade is, in order to fit the ques- 
tions to the trade. The best preparation for assembling questions on 
a trade is the acquirement of a large amount of knowledge concern- 
ing the trade. 

Ordinarily from three to five experts in the trade, as defined below, 
should be employed in the assembly process. Psychologically the 
reason for this is simple; what will not be thought of by one expert 
will probably be thought of by another, so that ultimately, by the 
addition of but a few questions to the list by each expert, a lengthy 
list of good questions can be secured. The questions obtained from 
Expert No. 1 should be first submitted to Expert No. 2, requiring 
him to answer them in the same manner as if the questions were 
being asked in the standardization process. The answers of Expert 
No. 2 will in a large sense indicate whether the questions submitted 
by Expert No. 1 are ready for standardization. Expert No. 2 will 
obtain an adequate idea of the type of question desired from thus 
seeing what questions have already been submitted. If Expert No. 2 
fails to answer any question submitted by Expert No. 1, this ques- 
tion should be carefully investigated with the aid of Expert No. 2 to 
discover whether it was a lack of information on the part of Expert 
No. 2 or a lack of clarity in the question which prevented his under- 
standing what was meant. In the latter case, it may be necessary to 
reword the question many times. 

Each question should thus progressively get better and better as 
submitted from one man to another in turn in the assembly process. 
If necessary, after the questions have been submitted to some two or 
three men with progressive revision by the examiner (preferably 
retaining the unstandardizable forms of questions) the questions 
should be taken back from the shop to the office and be typed. Then 
whenever successive experts fail to give the expected answer, the 
various alternative forms hitherto given can first be administered, 
finally ending by securing that man’s contribution to another revised 
form. At any time that the exact expected answer is not immediately 
obtained from a superior workman, the difficulty should always be 
investigated. The assembler must remember, however, that even 
the best experts should fail on some questions, for it is only by hav- 
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ing some very difficult questions that the expert can be picked 
out. ¥ 

Having secured, by progressive revision, an acceptable set of 
questions, the questions should be typed ready for standardization. 
Out of, say, one hundred questions secured from the first three ex- 
perts, at least forty or fifty of the best questions should have stood 
the test of revision and be acceptable for standardization. The final 
selection of this set of fifty questions is made by a final administra- 
tion of the questions to a “critical” examiner in the office, preferably 
a person who has had standardizing experience in various trades, and 
who has had some assembly experience on this particular trade. The 
answers to the entire set of questions, when this person attempts to 
guess all possible correct and incorrect, sensible and absurd answers 
to each question, combined with the joint judgment of the assembler 
and examiner as to whether an adequate sampling of the various 
aspects of the trade has been made and whether the questions are 
short enough to be quickly and easily administered, will be adequate 
enough for making the final selection of fifty questions for standard- 
ization. The critical examiner, upon being told the true expected 
answer, should attempt to guess synonymous or ambiguous alter- 
native answers. If, then, the assembler’s questions have been so 
worded as to pin the guesser to “I don’t know,” to the correct answer 
if known, or to an admittedly random guess, they are ready for typ- 
ing for standardization. It is preferable to arrange the questions in 
their seeming order of difficulty, the easiest first. Even a very ex- 
perienced assembler will often make very bad mistakes in his judg- 
ment of the difficulty of particular questions. However, a few easy 
questions, put at the beginning of the list, are desirable to set at 
ease the subjects used for standardization. 


STANDARDIZATION 


Standardization, according to any procedure, consists in admin- 
istering the entire set of standardization questions, say fifty in num- 
ber, to a group of tradesmen and non-tradesmen, of four classes of 
known ability, defined as follows: 

Novice: A person totally without formal trade experience. Used as a check on 
examinations to make certain that the answers to questions cannot be guessed by 
an intelligent non-tradesman. 

APPRENTICE: A beginner or learner in the trade, as determined by length of 
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experience in the trade, the kind of shop, and the foreman’s opinion of a man. This 
does not necessarily coincide with the formal four or five years period of appren- 
ticeship of the trade unions but recognizes the fact that apprentices of exceptional 
ability and two years experience may be more valuable workmen than some 
journeymen of less intelligence and a larger number of years of experience. In 
terms of experience alone, following the army practice, in no trade is more than 
five years experience required for a person of average ability to become a journey- 
man. The word “average” is to be noted here as recognizing the only point of view 
consistent with the facts of the distribution of human abilities. Concretely, this 
means that in answer to such questions as, “How long does it take to learn this 
trade?” one can consistently answer by stating how long on the average it takes a 
person of average ability to learn the trade. Statements of time required to learna 
trade have no meaning unless one considers human ability as a variable quantity 
and unless the particular degree of ability to learn is specified as well. 

JOURNEYMAN: A workman past the apprenticeship period, but not yet an 
expert. A practical criterion for journeyman is: not capable of working alone 
without directions or not capable of giving orders to others. A man with the 
requisite experience in years for an expert but not so classed by his foreman is to be 
considered as a journeyman. 

Expert: A man past a definite stated minimum number of years of experience, 
varying with the trade considered, and in addition so classed by his foreman. In 
the army work, a foreman of a shop was nearly always classed as an expert. A 
man capable of acting as a foreman, or capable of working alone on difficult jobs 
without directions from others, was ordinarily also classed as an expert. 


It can be readily seen that there are no sharp dividing lines be- 
tween these trade classes in just the same way as there are no sharp 
dividing lines between the normal and backward, or backward and 
feeble-minded in intelligence. The expert is but a higher degree 
journeyman; the journeyman but a higher degree apprentice. Even 
the line between tradesman and non-tradesman is not so sharp as it 
appears at first glance; a tradesman of but three days’ experience is 
but a novice for nearly all trade test purposes; and many persons 
may possibly be found, such as vocational guidance experts, mechan- 
ical engineers, and the like, who have a smattering of knowledge in 
many trades. There is no difference of “stuff” out of which these 
different classes of men are made. These classes are but arbitrary 
classes which are to be fairly well recognized in some trades, at 
least, as forming the promotional steps of a workman whenever 
these promotional steps are based upon trade ability and not upon 
mere number of years of experience. In some trades the expert may 
have a radically different type of work to do, at least a part of the \ 
time, than the journeyman; while in others the apprentice may have 


26 Trade Tests in Education 


a radically different type of work from the journeyman. The appren- 
tice bricklayer, so far as trade tests are concerned, is the hod carrier; 
the apprentice riveter, for the same purposes, is the “heater up.” 
These are artificial distinctions and may not be agreed to by the 
workmen in the trade. Where they occur, they are very valuable for 
distinguishing the various classes of men. In many cases it is diff- 
cult to decide in which status to put a given man; in such cases all 
possible evidence should be taken into account and the men classified 
according to the best judgment thus resulting. Evidently, to be 
scientific, this rating of men must be complete before any tests are 
given. The best practice, moreover, if it is possible thus to secure 
sufficient subjects for standardization, is to select only men who are 
cleancut examples of journeymen, apprentices and experts. 

Next we should consider the number of men of each such class to 
whom the questions should be administered in standardization. In 
the army, an effort was made at first to administer the standardiza- 
tion questions to twenty each of apprentices, journeymen and ex- 
perts and to forty novices. Later experience showed that even with 
the total resources of all workers in a great manufacturing district 
such as Newark or Pittsburgh to draw from, often not over a half 
dozen apprentices were to be found. The requirements in numbers 
were therefore changed as follows: 40 novices, 10 apprentices, 30 
journeymen, and 20 experts. These proportions conform more 
nearly to the proportions in which the various classes are to be found 
in industry. Not so large a proportion of experts are generally to be 
found in any one shop. In a single shop, making up tests for the 
employment office of that shop, it will often be necessary to test all 
men in the shop if time permits. If there are, say, two rooms and 
twice as many men as it seems desirable to test, the best results will 
be obtained if an impartial sampling is taken of the men in both 
rooms, such a sampling as is to be easily obtained by testing all men 
in Room 1 whose last names begin with A to J inclusive, and in 
Room 2 all men whose last names begin with K to Z inclusive. 

The order in which the various groups are tested is worth consider- 
ing. Preferably a few apprentices are to be tested first. A low per- 
centage of “passes” insures that the test is difficult enough to differ- 
entiate between different grades of ability. These should be followed 
by tests of a few experts. Ordinarily there should be no overlapping 
in total crude scores between these two groups. If overlapping 
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appears, the test must be critically examined, since it is then likely 
to be unsatisfactory. 

Following the administration of the questions to the men of differ- 
ent classes of ability the questions are each statistically evaluated, 
only those being retained for the final examination set which fulfill 
the requirements of (1) statistically differentiating between the 
different degrees of trade ability, a graded series of difficulties being 
used; (2) being administratively acceptable as questions which will 
readily be answered as expected by a majority of expert tradesmen. 


PICTURE TRADE TESTS 


Picture trade tests are constructed in almost the same way. Since 
they are but oral tests, making use of pictures in place of descrip- 
tive phrases or definitions, all the points previously mentioned 
in respect to oral trade tests apply to the picture tests. 

Care should be taken so to word the questions that the exact point 
of a picture referred to by the question shall be immediately recog- 
nized by the subject without requiring the examiner to point to any 
part of the picture. Care must therefore be taken that the pictures 
used for illustration be clearcut pictures. It is often possible to secure 
pictures, clipped from trade catalogs, which will clearly illustrate 
certain parts of machines and not others; in these cases, the ques- 
tions should refer only to the clear part of the picture, other pictures 
being secured to illustrate the other parts of the machine. Of par- 
ticular value are the pictures often found in catalogs, used to illus- 
trate special appliances of machines, in which the background of the 
machine is purposely faded out in the photographic process to make 
the desired part of the machine stand out clear and distinct. 

There is little advantage in having photographs specially made for 
a picture test. Ordinarily a very great abundance of good pictures of 
machines and parts may be found in trade catalogs. White ink, con- 
taining china white, is to be used for blotting out any undesirable 
lines, lettering or unessential backgrounds on the catalog pictures. 
The pictures used for the final series should not, in general, show 
any more details of the machinery than is necessary to illustrate 
the principles to which the questions refer. For instance, if one 
desires merely the name of a tool, one should select a picture which 
gives merely the outside outlines of the tool. In preparing catalog 
pictures for being photographed, care must also be taken to see that 
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photographs which have been originally photographed and devel- 
oped “light” or “dark” are arranged together if best results are to be 
obtained. 

The assembler of picture trade tests must usually know more of 
the trade than is required of the assembler of oral tests, since the 
former is required to judge largely for himself which pictures are 
best to use. This applies also to performance tests, wherein the 
assembler must be reasonably competent to judge whether a pro- 
jected task involves enough of the essential tools and operations of a 
trade to be a fair sampling of the tradesman’s skill and knowledge. 

Once assembled and revised in a manner identical with that used 
in revision of an oral trade test, the picture test is ready for stand- 
ardization. If both forms of the test are being constructed, much 
time may be saved, and a comparison of the respective reliabilities of 
the two forms of test may be secured, by standardizing both forms of 
the test on the same subjectsat the same standardization examination. 

Picture tests are not ordinarily made with the expectation that 
they will be better or quicker methods of examination than the oral 
method. Rather is their value to be sought in overcoming language 
difficulty or in checking up on suspected cases of coaching. To secure 
the same amount of information in regard to literate applicants’ 
trade abilities, the picture test usually requires more time than the 
oral test. The generally safe rule to follow, when in doubt whether 
to use a picture or oral test, is to use both. 


PERFORMANCE TESTS 


Performance tests are more like picture tests than like oral tests. 
The elements for a performance test are chosen in much the same 
way as elements, suitable for reproduction by pictures, are chosen for a 
- picture test, the only difference being that such tasks nust be chosen 
as will lend themselves to easily and objectively measurable scoring 
points. For measurement of the finished products a series of “limit” 
gages each varying from the preceding by thousandths or half- 
thousandths is preferable to the use of micrometers with their great 
possibilities for subjective errors of the scorer. Statistically, the 
ordinary limit gages of “dimension + .oo1” are as poor a measure of 
a product for test purposes as dividing all humanity into short, tall 
and medium people. Thus partial credits are allowable, logical and 
necessary for best results in performance tests, which is not true in 
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case of the oral or picture tests where the burden of partial credits 
should be thrown upon the assembler who is to be compelled to as- 
semble clear, understandable questions. 

In the progressive assembly of performance tests, it is desirable to 
make use of both apprentices and experts, in order to determine what 
scoring points will yield differentiation of trade classes. Any scoring 
point which yields much overlapping of these classes is likely to be of 
little value, although more leeway may be allowed here than in the 
case of either of the other two forms of test. The product must be 
changed and rechanged during revision until a scorable and diag- 
nostic product is discovered. Since a product is limited in its number 
of dimensions, it will be seen how difficult it is to devise a good per- 
formance test. Scoring points in regard to tools used, manner of 
holding tools, and the like,—deviating from the desirable method of 
scoring only on the finished product, which method does not require 
the services of an observer while the product is being produced,— 
must often be resorted to in this form of test, in order to get enough 
scoring points to differentiate adequately between different classes of 
workmen. The time required for the total operation must always be 
carefully taken and preserved, quality and quantity of production 
being the two generalized measures of the workman’s skill in fash- 
ioning his product. The norms are always to be given in terms of 
quality X produced in time Y, or some composite index involving 
both of these variables. This is not necessary in the case of either oral 
or picture tests as the tests are given without a time limit, emphasis 
being placed upon the workman’s “possession or non-possession” of 
the “essential” elements of trade knowledge and skill. Since the oral 
and picture tests have not been given with a time-limit, it is pos- 
sible that with them also, better results might be obtained by reading 
the questions at a definite, predetermined rate and requiring the 
applicant to write his answers within a given number of seconds. 

Copious notes, in addition to a minimum number of standard 
recorded observations of the standardization on workmen, should be 
taken in the standardization of performance tests. The best per- 
formance test is one in which mere measurements of the product will 
yield the desired differentiation; but if the final statistical evalua- 
tion should show that the workman’s selection of tools, manner of 
handling tools, etc., are required as scoring points, these will be val- 
ueless unless carefully taken for every workman used in standardiza- 
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tion. For the assembly process, twice as much time should be used 
for the performance tests as for the oral or picture tests. Several 
times as long will of necessity be taken for both standardization and 
scoring, not to mention a proportionately longer time in the admin- 
istration. In all trades save a very few which are almost wholly mat- 
ters of skill, such as that of automobile driver and stenographer, per- 
formance tests have yielded per unit of time spent in examination or 
original assembly and standardization but a fraction of the examina- 
tion value yielded by oral or picture tests. They have, from every 
viewpoint save that of dispelling an obstinate foreman’s prejudice 
that “a man should be tested by giving him a job to do,” but little 
claim to merit as tests for most tradesmen. 


THE WRITTEN TRADE TEST 


In the standardization process the administration of the questions 
to novices by means of written examinations proved highly successful 
in the army procedure. This is due, no doubt, largely to the fact that, 
for practical purposes, novices may be categorically classified as 
“knowing” or “not knowing” the items of information called for in the 
oral questions. If unable to answer them, they were seldom willing 
to make guesses. On the other hand, it has been found that trades- 
men are very likely to question the correctness of some of the ques- 
tions used, and it seems doubtful whether the method would have so 
high a degree of success with tradesmen as with non-tradesmen. 
Nevertheless, it is probably the case that much of the prejudice of the 
army camp examiners against a written method was the result of 
the reflection of the point of view of the practical man in industry, 
namely that “a man may be able to tell about his trade, but not be 
able to write about it.” One can readily see that this is but an exten- 
sion of the belief of such persons that a workman may be able to doa 
job, but not to talk about it. Statistics would probably show that 
there is a high correlation not only between trade skill and ability to 
talk about the trade, if the proper questions are used, but also be- 
tween trade skill and ability to write about the trade, if, again, the 
proper questions are used. 

The chief value in using any written trade test method, is, of 
course, the saving that results both in the actual examination time 
and in the time required to score the subject’s replies. One examiner 
can examine five hundred men by written methods almost as quickly 
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as a single man by oral methods. The scoring may be done with great 
rapidity by stencil and by clerical help. Again, if the questions have 
been adequately standardized before administration, a definite, pre- 
determined set of answers eliminates all subjective judgment on the 
part of the scorer. 

One further advantage, only slightly used in school examinations, 
is the possibility of examining large numbers of persons at one time, 
allowing all who are evidently well above the minimum basis, to pass 
without further question and reserving those who make low marks 
for further detailed individual examination by oral or other methods. 
Of all the above methods of trade examination, the one-word-answer 
form of test is probably most readily adaptable to vocational school 
use. The only material needed for their assembly is paper and pencil. 
The questions, either printed or mimeographed with blank spaces 
following each question for the answer, are almost self-administra- 
tive. The scoring is thoroughly objective and rapidly done by means 
of stencil. This form of test is being given a thorough trial in the 
army E. and R. schools, both in tests of vocational aptitude and as 
final examinations, or tests of proficiency. 

Another form of written test, easily adaptable to vocational school 
use because of its extreme ease of construction, is the multiple choice 
test. This test hasnot yet been givensufficient trial to have its merits 
actually determined. Among its obvious advantages are: 

(1) The elimination of all writing. (2) The administration of more 
questions per unit of examination time. (3) Perfect objectivity in 
scoring, which is done by stencil. (4) Less scoring time per question 
than by the recall or picture method. (5) The possibility of using 
questions not adaptable to the oral or picture method, of discriminat- 
ing between good and bad practice, and of making judgments as to 
the more preferable of two or more alternative methods, and the like. 

Some light on the relative merits of these various methods will be 
provided by the results of the following three experiments. 


THE BRICKLAYERS’ TEST 


An opportunity was given, through the courtesy of the Pittsburgh 
Bricklayers’ Union, to experiment with a written bricklayers’ exam- 
ination devised on the well-known multiple choice method.! 


1Chapman, J. C. and Toops, H. A. “A written Trade Test: Multiple Choice 
Method,” Journal of Applied Psychology, Vol. 3, No. 4, 1919, DD. 358-365. 
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The men were all examined in one large assembly hall. There was 
therefore no possibility of arriving at incontestable measures of each 
man’s trade status. Four questions requiring definite answers in 
regard to each man’s trade experience and trade leadership ability 
were typed at the top of each test blank, and were answered by all. 
These answers were subjectively evaluated independently by two 
examiners and the status thereby determined. It is not to be ex- 
pected that this method would result in a status comparable to the 
status given a man by his foreman, since there is present the element 
of a man’s personal judgment of himself, and this is known to be 
highly unreliable. Various methods of selection of questions were 
tried out. It is unnecessary to give here the detailed explanations of 
the procedure. The following principles seem to be JUSS a as a 
result of this experiment: 


1. A few extremely bad questions will not appreciably change the 
relative marks of men if a large number of questions is used. 

2. To secure the same differentiating ability this method of test- 
ing would seem to require a larger number of questions than with the 
corresponding oral form of test. 

3. The eighteen best questions, chosen as conforming to the ideal 
type of graph distribution of correct answers,! are considerably better 
in differentiating the known groups into their proper classes than the 
entire set of sixty-two unselected questions. 

4. Other things being equal, the larger the number of difficult 
questions in a written test, the better the test. Statistically this 
amounts merely to a restatement of the principle that the greatest 
reliability of individual scores is obtained when fifty per cent of a 
given test group pass and fifty per cent fail a given question. 

5. Too many easy questions, as also too small a total number of 
questions, tend to cause (a) novices and apprentices to occupy too 
much space in the total scoring range and (b) journeymen and ex- 
perts to “pile up” in the upper limits of the scoring points. It follows 
from the principles above enumerated that a large number of easy 
questions may, under certain conditions, be more generally efficient 
than a very much smaller number of difficult questions. 

6. The results from the novices, persons so chosen that they 
should have been able to answer many of the questions if such ques- 


1“The Making of a Trade Test,” Personnel, Vol. 1, No. 7, 1918. 
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tions are capable of being guessed easily, show that this type of 
examination cannot be “bluffed.” 

7. The tradesman are, however, unwilling to make random 
guesses in this type of examination if they are uncertain of the cor- 
rect answer. For this reason it seems that little is to be gained by 
scoring the test by any arbitrary scoring formula such as S=R—W. 
Especially when given with a work limit, as was this test is the 
discount for the guessing privilege unnecessary, since the relative 
ranks of the men remain the same, all questions being presumably 
attempted by all men. If, however, the test were to be given as a 
time limit test, a scoring formula might always yield better 
results than by counting as the score the Rights alone. The best 
theoretical discount, of course, is that discount which would be 
determined by solving for C in the scoring formula S=R—C- W, in 
which C is a constant, the ratio of the partial regression coefficient 
of W, divided by the partial regression coefficient of R, with S as an 
independent criterion of the measure of trade ability of the men 
tested. 

THE FARMERS’ TEST 


To further test the multiple-choice plan of written examination, a 
farmers’ test was similarly prepared. The one hundred and thirteen 
questions evaluated in standardization are shown below. 

This test represents the one hundred and thirteen questions left 
for evaluation after seventeen of the original questions had been at 
once discarded as a result of the administration of the questions to 
farmers, and later two more, Nos. 84 and 85, of the list of one 
hundred and fifteen originally retained. 

The numbers in the right-hand margin indicate the number of the 
four choices which is the correct answer as judged by Ohio farmers. 


FARMERS’ TEST 


Each of the numbered questions below can be made to read correctly when one 
of the four words in the parentheses is underscored. Underscore the one proper 
word which will make the sentence read correctly. Do not underscore more than 
one word. If you are not sure which word is right, make the best guess you can; 
you will be discounted if you do not answer every question. You may have all the 
time you wish but accuracy is highly important. The two samples are marked 
correctly. : 

A. The end-gate is part of a (barn, fence, wagon, buggy). 
Samples Eas 


B. The Berkshire is a breed of (sheep, horses, cattle, hogs). 
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INFORMATION QUESTIONS: 


The Babcock Test is used to determine (the health of an animal, the con- 
dition of the soil, strength of a solution, butter-fat of milk) . 

A hame is part of a (binder, tractor, harness, potato-digger) 

A scythe is usually driven by (steam-power, electric-power, horse-power, 
hand-power). ; 

A hay-tedder is feeds in esdue Ae uals hileadine, havin 

The Guernsey is a breed of (turkey, sheep, cattle, horse) . 

Hens usually set on (6, 10, 15, 20) eggs 

A cow gives milk from (2, 4, 6, 12) teats 

In driving a team, the command, “Haw,” means to (Gua to wthe jefe! turn 
to the right, go straight ahead, back up) . F 

Acid phosphate is a (drug, food, drink, fertilizer) . 

oe is a (feed, flower, tool, disease) é 
A whiffletree is a (plant, tool, animal, part of a wicoa 


. Roup is (part of a harness, a garden vegetable, disease of chickens: a 


stock food) : 
Gaps is a disease of (poulcey, aheen, cattle! see : 
The first crop planted in spring is (corn, oats, beans, buckwheatye 
An average yield of wheat per acre is (10, 18, 30, 50) bushels . 
A capon is a (chicken, fruit, garden-vegetable, field-crop) 
Corn is planted i in (March, May, June, July) 
Corn stover is used in feeding (chickens, cattle, hogs, les 


. Casein comes from (meat, milk, corn, fruit) 

. The deepest-rooted forage plant is Cimothy, Giecer alfalfa, ieaillet) 

. The most hardy fruit is the (apple, peach, plum, cherry) 

. Chess, or cheat, injures (oats, barley, rye, wheat) . : 

. A horse is usually broken to work at age of (2, 3, 4, 5) years . 

. A good yield of hay is (1, 2, 3, 4) tons per acre . 

. The longest strawed grain is (wheat, barley, rye, oats) . : 

. A good hand will husk (25, 50, 100, 150) bushels of corn in a day 5 
. The heaviest milking breed of cows is (Jersey, Hereford, Shorthorn, Hol- 


stein) . 


. Animals tat shew tie eves are Shores hoes calles doo 
. Asnath is the handle of a (plow, rake, fork, scythe) ae 
. Legumes are a (family of plants, part of a harness, fixtures in a sabe. 


attachment to a corn-shredder). 


. Aman and team should be able to care ae and cultivate Ge as ie. 80) 


acres of corn in a season. 


. One breed of hogs is the (erelord: Miers Nerina, Duron 

. Wheat is cut with a (mowing-machine, scythe, binder, rolling-cutter) 

. Acommon make of binder is (Iver-Johnson, McCormick, Ford, Racine) 
. A tool used in cultivating a garden is (sulky, drill, hoe, roller) 

. The kind of horse most used on a farm is (road, coach, draft, race) 

. A good cow feed is (straw, alfalfa, timothy-hay, oats) . 

. A good hog feed is (corn, fodder, sorghum, peas) 
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. A fowl used on most farms is (chicken, goose, turkey, pheasant) . . ° 
. A tool used in hay-making is (binder, mower, scythe, harrow) 

. Field corn ripens in (June, July, September, November) . 

. Winter wheat ripens in (March, April, July, October) . 

. In preparing ground for nee a farmer uses (harrow, seed drill, peeden 


subsoiler) 


. Hay should be cured (15- nutes 30- munutes I- awe T- ees) before 


being put in the mow. 


. A rolling coulter is part of a (arto Atnche eae on 

. A share is part of a (binder, harrow, plow, tedder) : 

. A doubletree is part of a (harness, wagon, sprayer, posthole- dicen is 

. Ditching can be done most easily in (spring, summer, autumn, inten 
. Cows are usually milked (2, 3, 4, 5) times a day : 

. A fat market hog will weigh about (50, 100, 250, 500) pounds 

. An average feeding steer will weigh (300, 600, 900, 1200) pounds . 

. A moldboard is part of a (scythe, engine, plow, corn-cutter) 

. A-swath is made by a (harrow, plow, binder, mowing-machine) 

. A pitman rod is part of a Cee ve corn-planter, roller, double- 


tree) 


. A disk is part ot a Weoliee poll. entree! hinder Towing machine) 

. A reel is part of a (mowing-machine, binder, roller, disk-cutter) 

. A drag (has shafts, has a tongue, is tongueless, has a pole) . . 

. A 6-foot binder is pulled by (2, 3, 5, 7) horses 

. Tomato plants should be planted about (1, 2, 4, 8) feet or in thew row 
. Vinegar is made from (grapes, blackberries, raspberries, apples) 

. Hog house doors should be at least (1, 114, 1%, 2%) feet high . 

. Sheep are best raised on (wet, level, hilly, weedy) ground 

. A field from which the wheat has been cut is called a (wheat- held. pas- 


ture, meadow, stubble-field) 


. There are (40, 60, 100, 160) square rods in an acre 

5 ON jobber i is sometimes used to plant (wheat, corn, oats, ee. 

. Aclevis is part of a (scythe, doubletree, jobber, clover-seeder) 

. Akingbolt is part of a (wagon, automobile, harrow, buggy) . 

. A plant grown both for pies and for cow-feed is (rhubarb, peas, pump- 


kins, turnips) 


. Post holes should be ue Aa, a, 7) eet eee 
. It requires (1, 2, 3, 4) bushels of oats to seed an acre ; 
. The first crop harvested in a season is satin oats, cine winter- 


wheat) 


. Pigs are usually weaned a at thea. age oe (a, a 10, 3) weeks 

. Pigs at birth weigh about (1, 3, 4, 5) pounds . : Sa 

. The incubation period for chicken’s eggs is (2, 3, 4, 6) ite : 

. A team plows (2, 3, 4, 5) acres of ground per day with a 14-inch walking 


breaking plow . 


. A week-old calf should es fed (2, i si ey ere ot cue Et a fea 
. Thumps is (part of a wagon, a garden implement, a symptom of disease 


in pigs, an unsoundness in the horse) 
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The most valuable forage for seek cows is (clover, timothy, oat-hay, 


alfalfa) i 
A threshing outfit will Whvesh: (Seo 1000, ee 4000) pusher Bf wheat 
loGe Mol) gee ein prencme term Uehcoe AAI Ee Ge) G16 od oe SG 


. Weeds most seriously ediuce he vied of (cone Gate hea hay) . 

. A manure spreader covers a strip of ground (3, 6, 10, 12) feet wide . 

. An ordinary smoothing harrow is (6, 10, 15, 18) feet wide : 

. Hilled corn should be planted (6 in., 8 in., 3 ft., 6 ft.) apart in chee rows 

. Plowing the soil for corn should be lone toa Heak of (2, 6, 10, 18) inches 
. To conserve the moisture in the soil a (drill, drag, harrow, cultivator) 


should be used immediately after plowing 


. A 1500-pound horse will eat (5, 10, 20, 50) pounds of hy a on 

. Corn should be cultivated to a depth of (1, 3, 7, 10) inches 

. A mowing machine cuts a path about (1, 3, 5, 8 feet wide) . 

. Acommon width of furrow is (6, 8, 10, 14) inches 

. Red clover ordinarily yields (1, 2, 3, 4) crops per season . 

. The best laying hens will lay (100, 200, 300, 400) eggs per year . 

. To produce good crops, wet soil should be ee subsoiled, frricaeea. 


dynamited) 


. A horse at work is fed Ga a 8, wa) quarts of wats aes a fost: Ae fees ot 
. Four-inch drain tile are usually (8, 12, 18, 24) inches long . 
. It requires (3, 5, 7, 9) acres of grass to furnish pasturage for a cow during 


the pasturage season 


. One of the birds that hops over Ne. ground j is obin, blackbird: crow, 


quail) . 


. It takes about (1, 4, 8, eS) pounds ot bindent éyane to bind: an acre vot Boos 


wheat . 

The tree of these foun iat ons its le earheces in une spring is (oak, oe 
maple, hickory) ties 

A veal calf should be marketed: at re age of a. 4, me o ie 

The first fruit tree of these to blossom in the spring is eT es ce 
pear, quince). Syl Ae ene ee 

Corn is planted toa depth of G 3. ie 8) faehes : : Behe 

A fattening steer will gain (%, 1, 3, 5) pounds a day in Weick é 

Three horses are hitched together by means of an eee pole, ee 
tree, doubletree) 

It takes (200, 500, 1000, Peon pounds of fori. per acre of eee 

It requires (4, 8, 14, 20) bushels of potatoes to plant an acre . 

A week old calf is fed (once, twice, 3-times, 4-times) a day . 

A peach tree should begin to bear at age of (1, 3, 5, 7) years . 

It takes (1, 2, 3, 4) bushels of wheat to sow an acre of good ground 

Apple trees should be planted (10, 20, 30, 50) feet apart . 

Cabbage plants should be planted (4, 6, 10, 16) inches apart in the rows 

In drilled wheat the rows are about (4, 7, 9, 12) inches apart . : 

A farm horse should be shod in intervals of (2-weeks, 6-weeks, 3- -months, 
6-months) . 

To destroy lice, hore may Whe dipped i ina Salavies for Gacineces ‘eurgent 
tine, lard, carbolic-acid) . ee hath Bs. ae Ge ee ee ne 
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The questions were administered to ten farmers living on Ohio 
farms and regarded as “successful” farmers; to twenty-one women 
novices living in the city, three of whom were born on farms; to 
twenty-four men novices, some of whom were college graduates and 
had lived for a long period of time on the farm, from five to eighteen 
years in their boyhood; to nine advanced students in a state college 
of agriculture, preparing to become high-school teachers of agricul- 
ture; and to thirteen high-school teachers of agriculture in the same 
state as the college of agriculture students. 

Taking the one hundred and thirteen questions, without further 
selection, and setting up the usual critical scoring lines (43) between 
novices and “semi-skilled” farmers; between “semi-skilled” farmers 
and “successful farmers,” all women novices, save one with previous 
experience of eighteen years on a farm, are rated as novices. None 
of the men novices received ratings above “semi-skilled.” Of the nine 
advanced students of the state college of agriculture, eight are rated 
by the test as “semi-skilled” and one as a novice; of the thirteen high- 
school teachers of agriculture, ten are rated as “semi-skilled” and 
three as “successful farmers.” Of the “successful farmers” all save 
one boy, included in the group, who was at the time attending high- 
school, are rated as “successful farmers.” The results thus show a 
remarkably good differentiation of farming ability as far as it can be 
judged from the measure, which is not, of course, fully adequate. 

The technical training of the college boys, subjected in all due fair- 
ness, it must be stated, to a somewhat different type of farming from 
that prevailing on the general farms of Ohio, was not sufficient to 
enable any of them to obtain ratings as “successful farmers.” That 
the difference is not due wholly to the difference in the section of the 
country is proved by the fact that the high-school teachers of agri- 
culture from the same state, and somewhat experienced in the actual 
farming operations of their vicinities, were able to make a higher 
average score, and one quarter of them to receive ratings as “success- 
ful farmers,” judged by the Ohio standard. We may infer from this 
that the questions selected were “practical questions” in the sense 
that one must have had actual farm experience in order to make high 
scores on the test. 

There are seventy questions of the set which no single one of the 
farmers missed. To the average man of the street such questions 
would be the ideal type of question. The reasoning would be some- 
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what as follows: If no single farmer missed any one of the seventy 
questions, then failure to pass any considerable number of the ques- 
tions would undoubtedly prove a serious lack of farming skill or 
knowledge. Philosophically, such owes its origin to the type of 
thinking characteristic of the trick question so popular with many 
foremen. Individual scores were computed on the seventy easiest 
questions—easiest for Ohio farmers—and the results graphed as 
usual. With the critical scoring points set up, farmers were separated 
from men novices, although there was little leeway in scoring points 
between the two groups. Inexperienced men novices and the experi- 
enced novices were here grouped together indiscriminately in scores. 
Thus, the set is not a bad examination, but could be very much im- 
proved by proper selection of questions. Undoubtedly it is only by 
having a very large number of questions that the rather good differ- 
entiation has been obtained. It must be remembered that even 
“trick questions” will differentiate the various trade classes if we use 
enough of them. The difficulty in industry with the “trick question” 
is that the foreman is prone to hire or not hire a man on the basis of 
his answers to one, or at most, two or three of these “trick ques- 
tions.” 

After the seventy easiest questions had been evaluated as above, 
the forty-three most difficult questions were similarly evaluated. 
It is astonishing to discover that the differentiation of groups 
secured by using the difficult questions is much inferior to that 
obtained from the easier ones. Men novices in general make scarcely 
any better scores than women novices; there is some overlapping of 
men novices with farmers; while four of the women novices made 
higher scores than the lowest farmer. This lack of differentiation is 
probably due to the small number of questions involved rather than 
to failure of difficult questions to secure the differentiation which 
difficult questions are usually expected to secure. The method gave 
very poor differentiation in spite of the fact that the questions were 
difficult. Examining our measure of the percentage of the total 
group who failed, it is a question whether the lack of differentiation 
with difficult questions is not perhaps due to the fact that the ques- 
tions, if graphed, would present unfavorable graphs. Without 
graphing, but merely from inspection of a chart on which the number 
of failures by known classes was tabulated, a set of forty questions of 
good graph form was selected. Counting all the men novices as one 
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class and as superior to women novices in regard to their effective- 
ness as farmers, since but few of the men were totally inexperienced, 
the forty questions yield an excellent differentiation of the three 
classes, and a perfect separation of “persons living on a farm” from 
“persons not living on a farm.” The results of this experiment con- 
firmed the conclusions obtained in the bricklayers’ written trade 
test, namely that (1) either a very large number of written questions 
must be used, or else (2) a careful selection on the basis of graphed 
questions must be made, and (3) when in doubt as to the choice be- 
tween two questions seemingly alike in other respects, it is prefer- 
able generally to select the more difficult question. 


A COMPARISON OF THREE FORMS OF WRITTEN TEST 


In order to determine the relative merits of three types of exam- 
ination method, the reactions of subjects to a test of general infor- 
mation were studied. There are two statistical aspects of the worth 
of any test: (1) its correlation with an independent criterion of the 
ability tested, and (2) its reliability, as measured by the correlation 
coefficient of reliability. Only the second of these two aspects has 
been investigated in this experiment. 

Fifty items of information, designed to sample a very wide range 
of general information ability, were assembled and arranged in three 
forms of test: (1) recall test, or army oral trade test, with the one- 
word-answer form of question,(2) recognition test, with the questions 
worded exactly the same as in the recall, but with five answers given 
to each question, only one of which was correct, (3) true-false test, 
the material being arranged to make use as far as possible of no new 
concepts. The tests are shown on the following pages. 


GENERAL INFORMATION TEST. RECALL 


Each question below can be answered with a single word or a number. 
Write the answer in the blank space following each question. 


Who was the first president of the United States? 
Ans. Washington. 

What color are ripe strawberries? 

Ans. Red. 


1. What letter designates the note on the bottom line of the staff in music? 


Ans. EE. 
2. What is the smallest fraction of a cent in which stocks are quoted? 


Ans. ¥%. 


Samples 


40 


24. 


25. 
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How many minutes does it take light to travel from the sun to the earth? 
Ans. 8. . 

By what sculptor was “The Thinker” made? 

Ans. Rodin. 

What was the nationality of William of Orange? 

Ans. Dutch. 


. Who, in American history, is known as “Old Hickory”? 


Ans. Jackson. 


. How many feet does an object fall from a position of rest in the first second? 


Ans. 16. 


. How many senators represent each state at Washington? 


Ans. 2. 


. What was the nationality of Rembrandt? 


Ans. Dutch. 


. What is width in inches of a hand by which the height of horses is measured? 


Ans. 4. 


. By whom was Jeffries defeated in the prize fight at Reno? 


Ans. Johnson. 


. Who founded the Mormon religion? 


Ans. Smith. 


. What wood was found best for aeroplane construction? 


Ans. Spruce. 


. Who said “I came, I saw, I conquered?” 


Ans. Cesar. 
How many teams are there in the National Baseball League? 
Ans. 8. 


. Who is the father of modern socialism? 


Ans. Marx. 
In what city was President McKinley assassinated? 
Ans. Buffalo. 


. What is the key of music which has no sharps or flats? 


Ans. C, 


. What American admiral first visited Japan? 


Ans. Perry. 


. In what country has the story of William Tell its origin? 


Ans. Switzerland. 


. What is the largest city in South America? 


Ans. Buenos-Aires. 


. How many feet are there in a fathom? 


Ans. 6. 


. What is the smallest planet in the solar system? 


Ans. Mercury. 
What acid is contained in vinegar? 


Ans. Acetic. 
For what American poet was the armchair made from the spreading chest- 
nut tree? 


Ans. Longfellow. 
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. In what river are the Thousand Isles? 


Ans. St. Lawrence. 
By what insect is the yellow fever germ carried? 
Ans. Mosquito. 


. On what mountain were the commandments received by Moses? 


Ans. Sinai. 


. What stick is generally used in golf when on the green? 


Ans. Putter. 


. From what animal is ambergris secured? 


Ans. Whale. 


. How many legs does an insect usually have? 


Ans. 6. 


. Who created the character of Ichabod Crane? 


Ans. Irving. 


. By whom was the “Angelus” painted? 


Ans. Millet. 


. What breed of dairy cow gives the greatest amount of milk? 


Ans. Holstein. 


. What chemical element is the basis of graphite? 


Ans. Carbon. 


. What is the name of the human bone which extends from the knee to the hip? 


Ans. Femur. 


. What Biblical character said, “Your people shall be my people?” 


Ans. Ruth. 


. Who discovered the Mississippi River? 


Ans. De Soto. 


. What mythological beauty was the cause of the Trojan War? 


Ans. Helen. 


. During what war did the charge of the light brigade take place? 


Ans. Crimean. 


. In what country is most of the world’s sisal crop grown? 


Ans. Mexico. 


. On how many hills was Rome built? 


ANS . 


. What wrestler is famous for the “scissors hold”? 


Ans. Stecher. 


. In what play does the character of Portia appear? 


Ans. The Merchant of Venice. 


. What instrument is used to test the strength of a storage battery solution? 


Ans. Hydrometer. 


. What state is called the “mother of presidents”? 


Ans. Virginia. 


. What breed of hen is best for egg production? 


Ans. Leghorn. 


. To what country does Greenland belong? 


Ans. Denmark, 
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49. What state contains a section known as the “Panhandle”? 
Ans. Texas. 

50. In what city is the Smithsonian Institute? 
Ans. Washington. 


GENERAL INFORMATION TEST. RECOGNITION 
Read each question carefully, and then underscore the one correct answer. 


How many inches are there in a foot? 

Samples } Ans. (8, 10, I2, 14, 16). 
Who was the first president of the United States? 
Ans. (Lincoln, Adams, Franklin, Grant, Washington). 


1. What letter designates the note on the bottom line of the staff in music? 
Aus, (A; GB, C). 
2. What is the smallest fraction of a cent in which stocks are quoted? 
Ans. (1/2, 1/3, 1/8, 1/5, 1/10). 
3. How many minutes does it take light to travel from the sun to the earth? 
Ans. (4, 8, 12, 16, 20). 
4. By what sculptor was “The Thinker” made? 
Ans. (Michael-Angelo, Bartholdi, Borglum, St.-Gaudens, Rodin). 
5. What was the nationality of William of Orange? 
Ans. (Dutch, Scotch, French, English, Irish). 
6. Who, in American History, is known as “Old Hickory”? 
Ans. (Johnson, Jackson, Lee, Lincoln, Boone). 


7. How many feet does an object fall from a position of rest in the first second? 


Ans. (8, 12, 16, 20, 24). 

8. How many senators represent each state at Washington? 
Ans.. (1, 2, 3, 4, 5) 

g. What was the nationality of Rembrandt? 
Ans. (German, Swiss, Belgian, Dutch, Austrian). 


10. What is width in inches of a hand by which the height of horses is measured? 


Ans. (4, 6, 8, 10, 12). 
11. By whom was Jeffries defeated in the prize fight at Reno? 
Ans. (Willard, Corbett, Fitzsimmons, Dempsey, Johnson). 
12. Who founded the Mormon religion? 
Ans. (Dowie, Young, Smoot, Smith, Penn). 
13. What wood was found best for aeroplane construction? 
Ans. (Hickory, pine, ash, spruce, hemlock). 
14. Who said, “I came, I saw, I conquered”? 
Ans. (Napoleon, Cesar, Washington, Cromwell, Grant). 
15. How many teams are there in the National Baseball League? 
Ans. (4, 6, 8, 10, 12). 
16. Who is the father of modern socialism? 
Ans. (Trotsky, Debs, Penrose, Tolstoy, Marx). 
17. In what city was President McKinley assassinated? 
Ans. (Buffalo, Chicago, St. Louis, Rochester, Milwaukee). 


18. 


19. 


25. 


26. 
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What is the key of music which has no sharps or flats? 
Ans. (A, C, D, E, G). 

What American admiral first visited Japan? 

Ans. (Farragut, Jones, Evans, Perry, Deucy). 

In what country has the story of William Tell its origin? 
Ans, (England, Switzerland, Spain, France, Italy). 


. What is the largest city in South America? 


Ans. (Rio-de-Janerio, Lima, Santiago, Valpariso, Buenos-Aires). 


. How many feet are there in a fathom? 


Ans. (4, 6, 8, 10, 12). 

What is the smallest planet in the solar system? 

Ans. (Venus, Saturn, Mercury, Neptune, Mars). 

What acid is contained in vinegar? 

Ans. (Acetic, citric, nitric, oxalic, carbolic). 

For what American poet was the armchair made from the spreading chestnut 
tree? 

Ans. (Whittier, Lowell, Tennyson, Longfellow, Bryant). 

In what river are the Thousand Isles? 

Ans. (Hudson, St. Lawrence, Delaware, Mississippi, Amazon). 


. By what insect is the yellow fever germ carried? 


Ans. (House-fly, cootie, flea, mosquito, tick). 
On what mountain were the commandments received by Moses? 
Ans. (Ararat, Everest, Aetna, Mount-of-Olives, Sinai). 


. What stick is generally used in golf when on the green? 


Ans. (Mashie, lofter, putter, brassie, cleak). 


. From what animal is ambergris secured? 


Ans. (Elephant, whale, walrus, camel, shark). 


. How many legs does an insect usually have? 


ANS.. (Ay 16s. 8,210jn12)- 


. Who created the character of Ichabod Crane? 


Ans. (Cooper, Dickens, Dumas, Irving, Stevenson). 


. By whom was the “Angelus” painted? 


Ans. (Corot, Bonheur, Millet, DaVinci, Velasquez). 


. What breed of dairy cow gives the greatest amount of milk? 


Ans. (Jersey, Guernsey, Ayreshire, Brown-Swiss, Holstein). 


. What chemical element is the basis of graphite? 


Ans. (Carbon, lead, mercury, zinc, iron). 


. What is the name of the human bone which extends from the knee to the hip? 


Ans. (Ulna, femur, humerus, cervical, clavicle). 


. What Biblical character said, “Your people shall be my people”? 


Ans. (Esther, Rachael, Rebecca, Ruth, Hester). 


. Who discovered the Mississippi river? 


Ans. (DeSoto, Balboa, Drake, Ponce-de-Leon, Cortez). 


. What mythological beauty was the cause of the Trojan War? 


Ans. (Venus, Minerva, Helen, Ceres, Juno). 
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40. During what war did the charge of the light brigade take place? 
Ans. (Napoleonic, War-of-the-Roses, Franco-Prussian, United States Civil, 
Crimean). 
41. In what country is most of the world’s sisal crop grown? 
Ans. (India, Brazil, Mexico, China, Turkey). 
42. On how many hills was Rome built? 
ANS. (35 59°73. 97 Ul) 
43. What wrestler is famous for the “scissors hold”? 
Ans. (Lewis, Stecher, Caddock, Roller, Demetral). 
44. In what play does the character of Portia appear? 
Ans. (King Lear, Twelfth Night, Julius Cesar, The Merchant of Venice, 
Henry the Eighth). 
45. What instrument is used to test the strength of a storage battery solution? 
Ans. (Hydrometer, thermometer, barometer, altometer, speedometer). 
46. What state is called the “mother of presidents”? 
Ans. (Massachusetts, Maryland, Ohio, New York, Virginia). 
47. What breed of hen is best for egg production? 
Ans. (Rhode-Island-red, cochin, wyandotte, leghorn, barred-rock). 
48. To what country does Greenland belong? 
Ans. Norway, Sweden, Denmark, England, France). 
49. What state contains a section known as the “Panhandle”? 
Ans. (Arizona, Texas, Louisiana, Kentucky, California). 
50. In what city is the Smithsonian Institute? 
Ans. (Boston, Philadelphia, Pittsburgh, Baltimore, Washington). 


GENERAL INFORMATION TEST. TRUE—FALSE 


Examine each statement carefully. If the statement is true, underline the word, 
“TRUE”; but if the statement is false, underline instead the word, “FALSE.” 


Camples { There are sixteen inches ina foot ...... TRUE FALSE 
\ Ripe strawberries are red’. » . . 2... 5... TRUE FALSE 

1. The letter G is the note which is on the bottom line of the 
staft in¢music-’ soo.) Ghee aba hae ne eee TRUE FALSE 

2. The smallest fraction of a Peent in Rvbich stocks are quoted 
IS: TUTO mee tee oe pone Se cree ee eee TRUE FALSE 

3. It takes light eight minutes to travel from the sun to ae 
Garth oP ys. tet legs (Sh es rae Seana » . IRUE “RLASE 
4. The statute of “The Tater’ was made by Bartholdi 7 Se URUESEAESE 
5. The nationality of William of Orange is Dutch . . . . . TRUE FALSE 
6. President Jackson is known as “Old Hickory.” .... . TRUE FALSE 

7. An object falls twenty feet from a position of rest in the 
firstisecond jp spon oo op see en ao RUE eALSE 
8. Two senators represent each state at Washinton Pa ea RUE eS ALESE 
9g. Rembrandt was of Dutch nationality ...... TRUE FALSE 


1o. A hand, by means of which the height of horses is mea- 
sured,:is)sixjinches in wiG thee spre ean ae TRUE FALSE 
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. Johnson defeated Jeffries in the prize fight at Reno . . . 
. Dowie founded the Mormon religion 
. Ash was found to be the best wood for zroplane con- 


epee) ey tees. te ta" 1, 


struction 


o> Keene), Kesey Wiel E ce” seh eh ey Uermts. ~ 0) aie; wferlia ue, ersee 


. It was Cesar who said, “I came, I saw, I conquered.” . . 
. There are ten teams in the National Baseball League . . 
. Marx is the father of modern socialism 
. President McKinley was assassinated in Rochester 

. The key of music which has no sharps or flats is C af hs 
. Perry was the first admiral to visit Japan ....... 
. The story of William Tell has its origin in Italy at 
. The largest city in South America is Rio de Janeiro. . . 
me Luere.are six feetrinvaiathom, jena iy) fe ae) eae 
. The smallest of the planets in the solar system is Venus . 
. Acetic is the acid which is contained in vinegar. . .. . 
. The armchair made from the spreading chestnut tree was 


madeorswonotelows eer path, oy Veen ck Ole lr ns 


. The Thousand Isles are located in the St. Lawrence River 
. The tick is the carrier of the yellow fever germ... . . 
. Moses received the commandments on the Mount of Olives 
. The putter is used in golf whenonthe green ..... 

. Ambergris is secured from the shark ......... 

mpAnvinsect UsuallyshasiolxleeShumura cis) teense cr Auer io ee 

. Irving created the character of Ichabod Crane . . . 

. The “Angelus” was painted by Bonheur . . . 

. Guernsey cows give the greatest amount of fale 

. Carbon is the chemical basis of graphite. ....... 

. The human bone which extends from the knee to the hip is 


calledbehestemur ais bie? Vans ie Sai dees reo ren Bees 


. Ruth is the Biblical character who said, “Your people shall 


ea ya DCO DICH coe te heey eect ies a ccs dase May soe ee Woe 


. Balboa discovered the Mississippi River. ....... 
. Helen was the cause of the Trojan War ........ 
. The charge of the light brigade took place during the 


ranco-Prussianwvate meatier ieee sae oe ins ap tae 


. Most of the world’s sisal crop is grown in India 

me Rome was) built:on-seven hills.-2 93. 2 2 2.0. - 

. The wrestler, Demetral, is famous for the “scissors pold: 
. Portia appears in the play, The Merchant of Venice 

. Abarometer is used to test the strength of a storage battery 


Oils, oS Choe Spee Ge. Ora es 


. Ohio is called the “mother of presidents.”. ....... 
. Leghorn hens are the best breed for egg production . . . 
= Greenland belongs to iceland) 0. see -) e 
. Texas contains a section known as the “Panhandle” 

. The Smithsonian Institute is located in Washington 
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The tests were given in all three forms to any one person at one 
sitting by the group examination method. Since results might con- 
ceivably be influenced by the order in which the forms were taken, 
all of the six possible combinations of orders were administered to 
different subjects of the same school classes by clipping together the 
test papers in the desired order of administration, hereinafter called 
“method.” The order of the methods is as follows: 


No. of 

Cases 
Method 1. Recall, recognition, true-false ...:........ rie 6 i) 
Method'2) | Recall true-false, recosnitiom |.) seinen eae nae eee 37 
Method'3. Recognition; recall) true-false™. ). <2. 4 =) 1s ue 14 
Wethodi4.. Recognition, true- talseyrecallay. a... seule ene ne: ieee 12 
Methodtsae Lrue-ialsex recall ;recocmition anu tire niente een ee 10 
Methodi6é. True-falsesrecognition,wecall) 2 2.) ee ess eee 12 
Total aa ret 1od sens oe co en ne ro ee 124 


The number of cases by methods is the same throughout, save in 
the calculation of average times taken on the tests where the fre- 
quencies are smaller because some subjects failed to record the time 
taken. The approximate times, taken to the last minute already 
recorded when a subject finished each test, were found by the simple 
expedient of the examiner marking on the blackboard the time 
elapsed at the end of every minute, the time of completion of each 
form being written on the test paper by the subjects. This procedure 
was not followed in two classes tested. The test was given by the 
work-limit method, accuracy and not speed being stressed in the 
directions. The subjects were summer school students at Teachers 
College. Since reliability is being investigated, sex differences or 
class differences need not be considered. The results are shown in 
Table I. 

On all methods, the subjects made the lowest average score on the 
recall form of test, a higher average score on the recognition form, 
and a still higher average score on the true-false form. The true- 
false form is decidedly too easy a test for university students, there 
being from 76 to 81 per cent of passes, according to the method. 

As shown by the average scores, the recall method profits by hav- 
ing one of the other forms given first. In methods 1 and 2, with recall 
given first, the average recall scores are 87 and 84 per cent of the 
average recognition scores; in methods 3 and 4, with recognition 
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TABLE | 


AVERAGE SCORES, AVERAGE INTERCORRELATIONS OF EACH TEST RESPECTIVELY 
WITH THE OTHER Two, RELIABILITY COEFFICIENTS OF HALVES 
or Test, AVERAGE TIMES IN MINUTES, BY METHODS 
AND BY TESTS 


Average Correlation| Reliability Co- 


Average Score of tests with other efficient of the Average time in 
on test two tests in halves of minutes 
each case tests 
iS 
pa (tee sels ies eae HERS 
$|3 =| 3 s| 3 a cee sss 
NOS a. = & xe s & = & ey ~ = © ~ s 
Merod| of |S] 8) S13] 8) se] 8) 8) 8) 3) e] g e2 
Number |Cases| © 2 es z ma aa ee G we S = § 
I 39 29.1 | 33.3 | 39.2] .872 | .670 | .8r1 | .724 | .487 | .405 | 7.8 | 5.4 | 3-4 28 
z 37 30.3 | 35.9 | 39.1 | .818 | .722 | .868 | .784 | .668 | .690 | 8.0 | 4.7 3.0 nag 
3 14 31.4 | 32.4 | 37-4 | .938 | .827 | .852 | .784 | .785 | .664 1] 5.5 8.2 3.0 I2 
4 12 35-6 | 36.0 | 39.6 | .926] .898 | .924 | .789 | .530|] .506] 5.4 | 5.6 | 3.7 4 
5 ae) 36.2 | 39.1 | 40.3 | .358 | .803 | .432 | .284 | .271 | .385 | 5.2 4.9 4.2 9 
6 12 35-5 | 36.8 | 38.2 | .968 | .740 | .762 | .695 | .675 | .604 |] 6.2 | 6.2 | 4.3 6 
Total 124 Average of six Total of Total 86 
Average 124 cases Average Weighted 
Tntercorrelations Time 


.813 .777 .775 | -448 .385 .340 6.9 5.6 3.6 


given first, the average recall scores are respectively 97 and 96 per 
cent of the average recognition scores, and the average recall scores 
are larger than in the previous two methods; in methods 5 and 6, 
with true-false given first, the average recall scores are respectively 
93 and 96 per cent of the average recognition scores, while the 
average recall scores are higher than in all but one of the four pre- 
viously mentioned methods. The true-false test is subject to little or 
no improvability through previous tests, and the recognition a lesser 
amount than the recall form of test. 

In the absence of a better criterion, we may say that that test is 
the best measure of general information, which, on the average, cor- 
relates highest with the other two tests. With this criterion, recall 
ranks first, recognition second, and true-false last. 
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The recall form, with fifty questions used in all three forms, will 
yield the highest reliability coefficients of all three forms. 

For further computations, all persons taking recall are grouped 
together, indiscriminately, irrespective of method; and likewise for 
the recognition and recall. From these total distributions the total 
reliability coefficients and total average times of the table are com- 
puted. 

Now, although for fifty questions, 1.e., the same number of ques- 
tions, done by subjects on recall, recognition and true-false forms 
respectively in an average number of minutes of 6.9, 5.6, and 3.6, 
the recall method is the most reliable, it does not necessarily follow 
that the method is the most reliable for the same average amount of 
time spent in examination. 

The average times show that, in the same time required to give 
one recall question, 1.23 recognition questions may be given and 1.92 
true-false questions. The question then is, “Will the added nu nber 
of questions which may be given in the same time be sufficient to in- 
crease the reliability of either recognition or true-false to a point 
exceeding the reliability of the recall?” To answer this question 
theoretically, we make use of the usual formula for reliability coeffi- 
cients of 2 forms of a test with ” forms, 

pie nN Ti 

(1) sh I ok (n == I)ru 
in which, 7y, is the reliability to be calculated, 7 is the reliability 
of the halves of the test, and is the number of multiples of the 
halves considered. Table II gives the 7 values, correlations of one 
25-question form with another 25-question form. 

When 1 = 2, we get the reliability of a 50-question form of the 
test with another 50-question set; z.e., Form A with Form B of the 
same test. The formula then becomes 

(2) 2h 

122 SS 
(recall) pee 77 
Solving formula (2) directly for recall, and formulae (3) and (4) 
below, respectively for recognition and true-false; 7.e., taking n of 
such value that equal times will be spent in examination, such time 
as is required to give 50 recall questions, or 6.9 minutes, we obtain 
the last horizontal row of Table II. 
(3) Pepin eG 
gnition) = + [2(1.23) —1]-rn 
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TABLE II 


COMPARISON OF RELIABILITY COEFFICIENTS OF THE RECALL, RECOGNITION 
AND TRUE-FaLsE Tests (ALL MetHops GrourED TOGETHER) 


Recall Recognition True-false 

Reliability (71) of halves. 

124 cases. Two forms of 

25 each. .448 385 340 
Reliability of two 50-question 

sets. Formula (2) 618 556 .507 
Average time in minutes to 

do 50 questions 6.9 5.6 3.6 
Number of questions per unit 

of recall time 1.00 te 1.92 


Number of sets of 25 ques- 

tions to get equal reliability 

of .618 2.000 2.602 3.140 
Reliability of Form A with 

Form B when 6.9 minutes 

examination time is used 618 .607 .664 


4) = 2 (1.92)-r1 
4 "2 (true-false) 1 + [2(1.92) —I}-ru 


This table yields the interesting result that, with all methods 
grouped together, with equal examination time, the true-false test 
is the most reliable, the recall less reliable, and recognition least 
reliable. 

In order to obtain an idea of the dependence of reliability upon 
the amount of time taken in an examination, we may plot the three 
equations involving reliability coefficients and examination time. 
This has been done in Fig. 28, where the ordinates are fy,, and 
abscissae 7, or rather, 2 reduced to time in units of the recall time, 
which is used as a standard of comparison for the other two, using 
the scale, 1 2’ = % (6.9) = 3.45 minutes, in which n’ is the average 
time in minutes required to do 1 multiple of 25 recall information 
questions. The numbers of questions, Q, done in ¢ minutes, are given 
respectively by the formulae: 

CS Via Recall, Q Alaa 

(6) Recognition, Q 8.83 ¢. 

(7) True-false, Q = 13.89 ¢. 


I 


¥ 
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These equations may be conveniently plotted as straight-line equa- 
tions on the same graph, using codrdinates on the right-hand margin 
of the graph. From the graph one can thus determine both the num- 
ber of questions and the examination time required to yield any pre- 
determined reliability coefficient. This graph holds good only for 
the range of ability represented by the test group of subjects and for 
the conditions of the experiment above outlined. 
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Fic. 28. THEORETICAL RELATIONSHIP OF RELIABILITY COEFFICIENTS TO 
EXAMINATION TIME. 


There is some question whether grouping all persons regardless 
of method or order of giving the test may not obscure some of the 
differences which exist in the reliabilities. The average times, for 
instance, show that if a group of persons given the true-false or the 
recall or recognition first, followed by either of the other two, the 
group tested tends to take more time on the test taken first than any ~ 
other group of persons given any other test first. Accordingly, the 
reliabilities have been computed, grouping together all the 76 people 
who took recall first, the 26 who took recognition first, and the 22 
who took true-false first. Evidently the results on these tests are 
not influenced by previous consideration of any other form of test, 
and so correspond as closely as may be to an actual recall or other 
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test as given alone and without any other. The following table 
results: 


Recall Recognition True-false 
ri) 25 questions -743 .689 561 
50 questions 
roo) Form A with 853 816 719 
{ Form B 

Average time in minutes 7.80 7.24 4.24 

: { Equal examination time 

™ ) of 7.80 minutes 853 826 824 


The reliabilities here are much higher. The added “conscientious- 
ness” when persons take the true-false first seems to make their aver- 
age time, which is not increased proportionately, such that the true- 
false is only about on a par with the recognition form when equal 
amounts of examination time are taken. The conclusions to be 
drawn from this experiment are: (1) When equal numbers of in- 
formation questions are given on the three forms of information 
test, the recall is always the most reliable, followed in order by 
recognition and true-false forms, but that (2) where equal amounts 
of examination time are taken on the three forms of test, the relia- 
bilities do not differ greatly. 


RELIABILITY OF VARIOUS TRADE TESTS 


The above technique allows us to compare the reliability coeff- 
cients of various trade tests based on different numbers of ques- 
tions. Reliability coefficients of the same magnitude are equal only 
when the range of ability is the same in the two cases being com- 
pared. Accordingly in Table III there is given a brief qualitative 
description of the group tested and their range of ability, as well as 
a brief qualitative description of the various tests, the number of 
questions in the test, the reliability of the halves of the test com- 
puted between the odd and even numbered question scores of the 


om ; 27 
test, the reliability of Form A with Form B of the same test, ———— 


Ih Sie Ti, 
and the reliability of Form A with Form B if fifty questions had been 
used in each test. The last column is thus the comparative measure 
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of the different tests, since it reduces all tests to the same number of 
questions. It is impossible with the data-at hand to compute the 
reliabilities with equal amounts of examination time taken. The 
author wishes to emphasize the point that all correlation coefficients, 
of which reliability coefficients are but one variety, must be inter- 
preted in terms of the range of ability; concretely, that since the 
range is limited to eighth grade students only, .795, the reliability of 
the girls’ trade school extension students on the one-word-answer 
written switchboard test, is as high a reliability as would be repre- 
sented by a much larger figure if several grades were to be included 
and the tests given to a larger range. Again, the best measure of an 
examination method is not the length of time spent in examination, 
but rather a composite of time spent in administration plus the time 
required for scoring. The time per person required for adminis- 
tration is often negligible as compared with the time required for 
scoring a test. 

From Table III it will be noticed that the reliability of all trade 
tests, measures of acquired learning, is uniformly high save in the 
switchboard operators’ test above mentioned, which is probably 
affected by the fact that only twenty questions were used in the 
examination, and in the test of progress in the university class in 
mental measurements likewise. The general information tests, or- 
dinarily rated as tests of intelligence, are in a distinctly lower class 
as regards reliability. 

This table presents the reliability coefficients of all the trade tests 
which the author has been able to collect up to the present time. 
Trade test methods give high reliability coefficients, provided 
enough questions, and sufficiently difficult or rather well scaled 
questions, are used. Forty selected difficult questions on the farm- 
ers’ recognition test gave a higher reliability coefficient than one 
hundred and ten unrevised questions. Selected, or revised ques- 
tions on the soldiers’ mechanical test gave a higher reliability than 
the unrevised questions. The one-word-answer form of test gen- 
erally gives higher reliabilities than the recognition or true-false 
forms. 

SCHOOL EXAMINATIONS 


Because of its high reliability and ease of administration and 
because it embodies all the superiority of the recall form of exam- 
ination over the recognition form, the one-word-answer method is 
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Group Tested 


Eyelet 
Machine 
Tenders 


Bricklayers 
Standardiza- 
tion Group 


Girls’ Trade 
School Exten- 
sion Students 


Farmers and 
University 
Students 


Men and 
Women 
Summer 
School 
College 
Students 


Men and 
Women 
College 
Students 


Soldiers 


No. of Cases 


Range of Ability 


Kind of Test 


No. of Questions in 


of 50 


basis 
questions, Form A 


with Form B 


14 


14 


61 


61 


271 


223 


Three weeks 
to one year 
of experience 


Apprentice to 
expert inclu- 
sive4A,5]J, 
5E. 


Alleighth 
grade pupils. 


8 women 
novices. 12 
men novices. 
ro farmers. 


“Above  col- 
lege entrance 
require- 
ments.” 


Graduate stu- 
dents in col- 
lege. 


Vocational 
courses E. & 
R. School. 


One - word - answer. 
Eyelet machine. 
Oral trade test. 


Army oral. One-word- 
answer. Army oral 
bricklayers test. 


One - word - answer 
written. Switch- 
board oral test. 


Multiple choice (4 
choices) written 
questions. Farmers’ 
test. 


General information 
test. All methods 
combined. 

Recall. One - word - 
answer. 
Recognition of cor- 
rect answer. 5 al- 
ternatives. 
True-false. 


School test in mental 
measurements. True- 
false. 


Same in one-word- 
answer form. 

Use of tools unrevised 
picture mechanical 
test. 

Revised picture me- 
chanical interest 
test. 


38 


20 


Al rio 


50 


50 


50 


20 


20 


46 


45 


.929 


-436 


.926 


-899 


957 


340 


+25 


+50 


-796 


-964 


.607 


.962 


947 


979 


.618 


-556 


+507 


-40 


.667 


886 


920 


-985 


-795 


+947 


958 


-953 


618 


-556 


+507 


‘625 


+833 


804 
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desirable in school examinations. The one-word-answer method can 
easily be adapted to school examinations, especially in those subjects 
which have a high information content: all types of science, physics, 
chemistry, botany, zoology, vocational courses, and technical subjects 
suchas mathematics, statistics, mechanical drawing, logic. Any study 
which possesses a large number of terms peculiar to such studies, 
paralleled by “trade terms” in the trade, lends itself admirably to 
this method. 

A physics examination, designed to illustrate the ease with which 
such questions may be written in a science subject, is herewith 
reproduced. 


GENERAL PHYSICS EXAMINATION 


To illustrate the trade test form of question 


1. What do you call a minute particle of matter which is made up of atoms? 
Ans. Molecule. 
2. What Physical property of iron allows it to be drawn into a wire? 
Ans. Ductility. 
3. What do you call the product of the mass of a body multiplied by its accel- 
eration? 
Ans. Force. 
4. What physical property of a body tends to keep it in motion whence once set 
in motion? 
Ans. Inertia (momentum). 
5. What do you call one-half the product of the mass of a body multiplied by the 
square of its velocity? 
Ans. Kinetic-energy. 
6. What is the C-G-S unit of force? 
Ans. Dyne. 
7. What is the diffusion of a fluid through a membrane called? 
Ans. Osmosis. 
8. Who discovered the law of falling bodies? | 
“Ans. Newton. 
y. How many feet per second of acceleration is given toa une body by gravity? 
Ans. From 32.00 to 32.16. 
10. What do you call a glass tube bent so as to drain water over a point higher 
than the surface of the water? 
Ans. Syphon. 
11. How much is the weight of one cubic centimeter of water at its greatest 
density? : 
Ans. I-gram. 
12. If the volume of a given weight of gas at constant temperature is doubled, 
how many per cent is the pressure decreased? 
Ans. 50. 


13. 


14. 


15. 


16, 
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18. 


19. 
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22. 


23. 


24. 


25. 


26. 


Age 


28. 


29. 


30. 
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What property of a tone is determined by the vibration rate? 

Ans. Pitch. 

What do you call it when rays of light are bent by going through a prism? 

Ans. Refraction. 

What shape of mirror reflects parallel rays of light to a focus? 

Ans. Concave. 

What are the first three colors of the primary spectrum, in order, at the long 
wave length end? 

Ans. Red-orange-yellow. 

At what Centigrade temperature does water freeze? 

Ans. Zero (0). 

What do you call the variation of a magnetic needle from a true north and 
south line? 

Ans. Declination. 

What kind of electricity is produced by friction? 

Ans. Static. 

What is the unit of electrical current called? 

Ans. Ampere. 

What other temperature scale is there besides the Fahrenheit a Centigrade 
scales? 

Ans, Reaumur. 

What do you call the force which makes a body fall to the earth when it is 
dropped? 

Ans. Gravity. 

What do you call a space from which most of the air has been pumped? 

Ans. Vacuum. 

What class of substances is generally easily volatilized? 


‘Ans. Liquids (fluids). 


What liquid is generally used in a thermometer for measuring temperature 
of more than forty degrees below zero? 

Ans. Alcohol. 

What do you call the ratio of the weight of a substance to the weight of the 

same volume of water? 

Ans. Specific-gravity. 

What is the unit of electrical power called? 

Ans. Watt. 

What is the unit of mechanical power in F.P.S. units called? 

Ans. Foot-pound. 

How many watt-hours in a kilo-watt-hour? 

Ans. 1000. 

What do you call the product of a force multiplied by the distance through 
which it moves? 

Ans. Work. 

What physical property has a body which causes it to rebound when dropped 
from a height upon a solid surface? 

Ans. Elasticity. 
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What do you call it when a sound wave has been reflected from a building and 
comes back to you as a distinct word? 

Ans. Echo. 

What is the name of the hypothesis which says that no force is ever created 
anew or ever destroyed? 

Ans. Conservation-of-energy. 

What do you call the force which holds together two molecules? 

Ans. Cohesion (adhesion). 

What is the force called which holds a soap bubble together? 

Ans. Surface tension (capillary attraction). 

What do you call it when water takes up a quantity of gas? 

Ans. Absorption. 

What do you call that part of a body about which, if balanced, it would be 
free to revolve in any direction with equal ease? 

Ans. Center-of-gravity. 

What do you call half the swing of a pendulum’s arc? 

Ans. Amplitude. 

How many foot-pounds in a horse-power? 

Ans. 33000. 

Who formulated the hydrostatic law which says that liquids transmit pres- 
sures equally in all directions and at right angles to the surface? 

Ans. Pascal. ; 

If the pressure on a block of wood two feet below the surface of water in a 
tank is one pound, how many pounds will it be if placed four feet 
below? 

Ans. 2. 

A block 2 cubic centimeters in volume weighs 8 grams in air; how many grains 
will it weigh in water at 4 degrees Centigrade? 

Ans. 6. 


. How many feet will an ordinary suction pump lift the water? 


Ans. From 27 to 32. 

How many feet per second is the velocity of sound in air? 

Ans, From 1080 to 1150. 

If a string four feet long vibrates twenty times per second, how many times 
per second will a string eight feet long vibrate if under the same 
general other conditions? 

Ans. 10. 

What is the instrument called which is used with a prism to measure the 
wave length of light? 

Ans. Spectroscope (spectrometer). 

What do you call two antagonistic colors which produce gray when mixed on 
a color wheel? 

Ans. Complementary (supplementary). 

What do you call the force which makes the governor on a steam engine act? 

Ans. Centrifugal. 

What is the phenomenon called when one electric current in a wire sets up a 
current in a neighboring wire? 

Ans. Induction. 
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50. What do you call it when hydrogen collects on the positive plate of a battery 
and thus stops the flow of current? 
Ans. Polarization. 


51. What is the decomposition of water by the electric current called? 
Ans. Electrolysis. 

52. What do you call the galvanometer which is used with an ordinary magnetic 

compass needle to measure electric current? 
Ans. Tangent. 

53. What kind of electricity is it called when electricity is generated by heating 

two different metals connected in series? 
Ans. Thermo (thermo-electricity). 

54. What do you call the ratio of the amount of heat required to raise a given 
weight of metal one degree Centigrade to that required to raise the 
same weight of water one degree? 

Ans. Specific-heat. 
55. What do you call the amount of heat required to raise the temperature of one 
pound of water one degree Fahrenheit? 
Ans. British thermal unit (B. T. U.) 
56. What, besides the volume of gas in a container, determines its pressure? 
Ans. Temperature (heat) (warmth). 


One may note that not only informational questions, but ques- 
tions involving the application of principles can be easily adapted 
to this form of test. In physics the use of formulae as a means of 
expressing physical laws is very common. One aim of most physics 
courses is to teach the students to be able to solve these equations by 
the simple process of substitution in the equation of values deter- 
mined from experiment, obtaining therefrom one unknown term of 
the equation. In an examination there is little merit in requiring the 
solution of equations which will involve decimal numbers. By using 
data which are in integral numbers and which are multiples of quan- 
tities which it may be necessary to divide, a test can be quickly made 
of the pupil’s ability to apply the physical principles involved. Take, 
for instance, the problem, “What Fahrenheit temperature is 20° 
Centigrade equal to?” In order to transform 20° Centigrade into 
Fahrenheit temperature, the pupil has merely mentally to perform 


the multiplication of 2 x 20 and then add 32°, giving an even quan- 


tity as the answer. It seems reasonable to suppose that the majority 
of students who can mentally perform that operation would also be 
able to perform the operation with the aid of a paper and pencil if 
22° rather than 20° were the Centigrade temperature. Whether such 
is generally the case is not known. The relative value of the two 
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types of tests is one which can be solved only by the method of cor- 
relation. When evaluating the relative merits of the two kinds of 
problem one must also take into account the fact that possibly two 
or three questions of the trade test form can be given in the same 
time as one of the other form. The problem in the last analysis is 
then “Will 2 questions of the trade-test form, in the same amount of 
examination time, give us more or less knowledge of a pupil’s ability 


in physics, than will z questions requiring extended arithmetical 


operations to be performed with the necessary aid of paper and 
pencil?” Of course the use of paper and pencil in computing 9 x 20° 
should be allowed if the pupil needs them. It is also a question 
whether this mental type of computation should not be taught in 
physics. In many of our courses in science and mathematics it is 
possible that a great many problems which can be solved mentally 
or with little effort would result in clearer insight into the principles 
and the uses of the formulae involved, than would a few lengthy prob- 
lems involving complicated arithmetical operations. Possibly this 
point is one of the main differences between ordinary school teach- 
ing and the training in the trade which a boy gets in industry itself. 
In industry a boy learns to do fractions by using a chart which con- 
verts the fractions into decimals. The commercial statistician, rec- 
ognizing the fallibility of “long-hand” methods, uses a calculating 
machine. 

Another point to be noticed in such examinations is the fact that 
in numerous instances the formula itself can be given. This again is 
the practice followed in industry. Undoubtedly the most valuable 
commercial statistician is the one who can at a moment’s notice 
write correctly any formula which he needs. The expert statistician, 
however, whenever complicated formulae are involved, resorts, as 
does the machinist in the machine shop, to his published handbooks 
of formulae, tables, and the like. A commercial physicist who could 
remember all his formulae would undoubtedly be able to do a larger 
amount of work in a day than one who had continually to resort to 
books, especially if the formulae were to be found in numerous books 
rather than in one hand-book. Such attempts to justify imposing 
routine upon a pupil in the study of any science are unworthy 
attempts at an outworn theory of discipline. They belong to the old 
puritanic philosophy of “discipline for discipline’s sake, work for 
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work’s sake,” and do not fit in with the modern philosophy of 
letting a machine do the work and reserving the more costly 
human machine for work which requires planning and exec- 
utive ability. 

One of the great barriers to securing large amounts of data on 
school practice, as a basis for revising school administration and 
school procedure, is the large amount of time required to give, score, 
and evaluate the tests. The trade test form of question, if it can be 
effectively applied to school subjects, has much to be said in its 
favor on this score. The above physics test clearly indicates the 
possibility of quickly examining a student on a very great many 
elements of the course pursued. Testing on many elements is a much 
more reliable method of securing an accurate evaluation of the indi- 
vidual’s ability than testing on only a very limited number, as is 
done where a composition is written in response to each question of 
the examination. The value of the one-word-answer method in edu- 
cational work remains of course to be proved. On the point of time 
taken to give the test and to score the results, we may state at the 
outset that a very great advantage can be secured, since the tests 
can be quickly scored by stencil. 

The use of the underscoring, or multiple choice method, of giving 
an examination may be considered on the basis of ease of scoring. 
This method is even easier to score than the one-word-answer 
method. The value of a test, however, should not be sacrificed 
merely for the ease of scoring. Using the underscore method, a pupil 
cannot make use of the independent thought which he must use 
when answering the recall or trade test form of question. Many 
more items of information can be touched upon by using the under- 
score method than can be tested in the same time with the one-word- 
answer form of question. Certain types of information may be easier 
to present with the one type of examination than with the other. 

Another point of practical consideration is that the one-word- 
answer form of question cannot be used very often in the recitation 
itself. It is difficult to form good trade test questions merely on the 
spur of the moment, and the same difficulty as is encountered in the 
trades applies here also. If only seventy-five good trade questions 
can ordinarily be discovered in a trade, then certainly not that num- 
ber will be found in the ordinary amount of subject matter covered 
by a class in physics in a single recitation. It is possible to multiply 
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examples of trade test applications of formulae, for instance, and so 
occasionally to obviate this difficulty. . 

Care should be take in using the trade test form of question that 
the teaching be not made to conform to the arbitrary standard set 
up by the trade test. Teaching would become rather artificial if the 
teacher always stressed the child’s giving the short answer form of 
replies which the trade test demands. Teaching would become life- 
less if the aim of the teacher were directed towards merely preparing 
the pupils to answer trade test questions. This applies, however, 
with equa] force to any measure of an educational product; the 
teaching always will become very lifeless if the teaching is directed 
towards preparing pupils merely to pass tests. There are some 
obvious exceptions, however; in the case of arithmetic or any such 
mathematics as the formulae of physics, the educational aim should 
certainly be to enable the pupil to use such fundamental operations, 
to solve equations with ease; in other words, to use his acquired skill 
in exactly the same manner as is required in the trade test question. 

The one-word-answer form of question can be used to stimulate 
the pupil to definiteness of expression. Both the composition type of 
examination question and the trade test form should be used, per- 
haps alternating the one with the other. The former encourages 
clarity of expression in composition; the latter, definiteness. The 
latter type of ability is perhaps demanded in industry to a greater 
extent than the former. The leading question, a type of question 
which indefinitely suggests the line of approach of the student, will 
continue to be used by the teacher as a means to stimulate the child’s 
curiosity and lead him on to rely on his own ability. The picture 
trade test method, essentially an oral method, is probably sugges- 
tive of some possible school examination methods such as have hith- 
erto not been attempted. The picture lathe hand test shown on 
pages 8-14,if made easier than it is now, could be adapted to written 
examinations in that subject in trade schools. The picture form of 
examination will also readily adapt itself to the teaching of schoo! 
subjects. 

Among the specific criticisms of the use of the one-word-answer, 
the picture, or the multiple choice tests in examining in school trade 
_ subjects is the often recurring one that such tests are tests of infor- 
mation mainly and do not require independent thought and organ- 
ization of subject matter. It is true that, in any trade examination 
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made up on these methods, a casual inspection will reveal a seeming 
preponderance of “informational” questions. This objection is mini- 
mized by the consideration,that the importance of “thinking” in 
doing many of the routine tasks of industry has probably been over- 
estimated. The most successful army oral questions were found to 
be the questions which seemingly best sampled the everyday habits 
and knowledge of the tradesman. In present-day industry, the 
workman’s task and the directions for performing it are worked out 
for him in advance; he relies largely on charts and mechanical devices 
for such “figuring” as is required; moreover, methods of scientific 
management and specialization in industry are continually breaking 
up trades into series of jobs, each of which is so simple as to be per- 
formed adequately by persons of only fair intellectual or mechanical 
ability. 

Finally, the whole question of whether or not many informational 
questions are desirable is one which will ultimately be settled only by 
the correlation resulting from experiments in which both “informa- 
tional” and “thought provoking tests” are tried with the same people. 
It must be remembered that thinking takes much time, while writing 
immediately recalled information takes much less time per question. 
The question then is, “For a given amount of examination time, will 
X thought provoking questions or (X +Y ) informational questions 
give higher correlations with demonstrated ability in performance 
on the job?” These methods are very apparently of limited value 
in the recitation, upon which should be thrown the chief burden of 
teaching pupils to think clearly. 

Psychologically, there is no reason why the results of a com- 
plicated thought process cannot be indicated by a simple one-word- 
answer response on the part of the subject. The mental arithmetic 
“reasoning” problems, to be found in great abundance in any ele- 
mentary arithmetic text, generally fulfill the major requirements of 
a good one-word-answer question. Few people would maintain that 
these are not thought provoking. It merely requires more skill on 
the part of the question maker to write “thought provoking” ques- 
tions than to write “informational” questions. 

All these methods are subject to the danger of preparation by 
coaching. The one-word-answer form of test is probably more sub- 
ject to this difficulty than the picture form. Certain forms of psy- 
chological tests are equally subject to this danger. The obvious 
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solution to the difficulty lies in having so many questions of each 
of the different forms that alternative tests may be provided. 

The tradesman in industry has seldom had an incentive for coach- 
ing up on any form of trade examination; in trade school procedure, 
examinations are expected as a matter of course. In the trade school 
the same material may readily be formulated in time into four forms 
of test, the recall one-word-answer, recognition or multiple choice, 
true-false, and picture, in order to overcome undue influence of 
“coaching for examination.” The coachability of a test is probably 
in inverse proportion to the number of thought provoking questions 
included, obeying thus the same law as applies in general to all ex- 
amination methods. 

Finally we may dismiss with but little discussion the possible 
objection that the acquisition of trade knowledge is not the aim of 
the vocational school. The present avowed aim is the preparation of 
youth to earn a livelihood in the vocations. We have before indi- 
cated the growing need in industry of the workman possessed of a 
goodly number of specific habits, skills and knowledges, and the 
growing tendency to simplify trades to such an extent that less in- 
telligence and less “mechanical ability” is required for success on the 
job by the majority of workers. 


CHAPTER III 
TESTS, OF TRADE CAPACITY OR PROMISE 


The possibility of predicting a person’s probable success at a job 
depends upon a relatively fixed condition of the job, a knowledge of 
the requirements of the job in the way of qualifications of its workers, 
and methods of measuring human qualifications and evaluating 
these in terms of degrees of success. Historically there have been 
three stages in the methods of attacking the problem, each of which 
has been characterized by developments in the methods of measur- 
ing the job on the one hand, and the worker on the other. 

In the first stage, which we may call the “mental type” stage, it 
was believed that success at a given job was possible only for per- 
sons of a given mental type. Efforts were accordingly directed 
towards determining the mental types best suited to given jobs, and 
toward determining to what mental type a given person belonged. 

This stage gave way to a second stage in which was attempted an 
analysis of the lives of successful and unsuccessful men, in order to 
discover what psychological or physiological trait or traits might be 
responsible for their success or lack of success. These methods were 
largely a failure. Statistically, the difficulty with them is that any 
one such single trait is very likely to have only a very low correla- 
tion with job success. On the side of measuring the job, efforts were 
made to classify jobs according to the predominant traits of char- 
acter and mental ability necessary for success on the job. 

This second stage then merged into the third, in which, not single 
traits, but combinations of traits, are considered responsible for the 
degree of success on the job. The development of tests of general 
intelligence marks the turning point in this stage. The general 
intelligence tests were found to be good predicters of general school 
success of school children, and to be of value in detecting children 
of abnormal mentality—sub-normal or super-normal ability to 
profit by the instruction received. From this it was but a step to 
infer that general intelligence scores would be found very significant 
in placing persons properly in industry. We are at present in this 
third stage but rapidly entering a fourth, in which the emphasis will 
be placed upon the development of statistical methods for the 
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evaluating into one composite score of fitness for the job all va- 
rieties of facts about an individual: his intelligence, parentage, 
physical capacity, formal education, informal trade training in 
mechanical manipulation of his environment and so on. Evidence 
has been accumulating rapidly in the last few years to point to the 
fact that individual traits may singly correlate but slightly with job 
success, but that a composite of such traits may correlate rather 
highly with job success. The method of partial correlation enables 
one to make the best possible mathematical prediction (assuming 
linearity of regression lines) of job success from a number of vari- 
able measures or tests.| The statistical requirements for a composite 
test of high predictive value are that the correlations of the tests 
with the criterion of success be high in relation to the intercorrela- 
tions among the tests themselves. Furthermore, there is coming to 
be stressed the fact that the reported correlation of the composite 
test score with the independent measure of job success can only be 
interpreted in the Jight of the range of ability covered by the test 
subjects; or, concretely, that a correlation of .60, where only eighth 
grade children are involved, indicates as good a composite test as 
would be shown by a much higher correlation if children from all 
the grades were included. 

On the side of measuring the job, there has arisen, just at the 
transition point from the third to the fourth stage of the develop- 
ment of human measurement, the method of job analysis. In its 
present best developed stage, the job is described in terms of con- 
crete job processes or equally concrete and measurable human quali- 
fications. Already we have heard hints of the possibilities of develop- 
ment of methods of measuring jobs by a scale of composite com- 
parable measures for all jobs, similar to the scales already being 
developed for human measurement. Fitting the worker to the job 
is only half of the larger social problem involved in vocational guid- 
ance. In many cases, jobs should be modified to suit workers. Such 
modification will tend to come about automatically as soon as the 
job analysis has caught up, in its process of development, with 
human measurement and entered likewise into the fourth stage. 

The names of many experimenters are connected with the develop- 
ment of present methods of predicting school success by the use of 
composite scales. The application of like methods in predicting 
industrial success has followed the former, and hence has fewer 
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names associated with it. Mention might be made of the names of 

Thorndike, Scott, Thurstone, Link, Burt, Marcus, and Otis. 

| Some six years ago Dr. Thorndike devised and put into use a 

~ series of intelligence tests for the selection of clerical workers for the 
Metropolitan Life Insurance Company. Among the tests were 
tests not only of general intelligence but of abilities similar to those 
employed by clerical workers in their work. The examination, con- 
structed in alternative forms in order as far as possible to overcome 
coaching difficulties, is reported to be giving successful resultsafter 
six years of trial, and yielding correlations with job success of 
probably Over .65._ 
Y Dr. Scott has devised for a number of firms! comets scales for 
— the selection of salesmen and clerical workers. His tests have often 
included the use of some form of group judgment or rating scale of 
important | ‘character qualities presumably not tested by the psy- 
chological tests) 


| Drx Ett l' reports many porreirtons sufficiently high to justify 

_ the belief that composite scales of various tests, selected on a “trial 
and error” basis, may be used to predict the chance of success in 
many industrial jobs, chiefly of a repetitive motor or manual 
type.)| .- 

Dr. Burt ? pore on the success of weighting tests by the method 
of partial correlation, using different weightings of the same tests 
for different jobs, by means of which the probable degree of success 
of a given worker in the rubber industry may be predicted for a 
number of jobs. Essentially the same technique was developed by 
Dr. Thorndike during the war, for rating members of the S. A. T. C. 
for different branches of the army. 

Mr. Marcus? has shown that a composite of psychological tests 
may be more efficient in predicting the probable success of certain 


1Link, H. C., Employment Psychology, 1919. 

2 Burt, H. E., “Employment Psychology in the Rubber Industry,” Journal of 
Applied Psychology, Vol. 4, No. 1, pp. I-7. 

3 Marcus, L., “Vocational Selection for Specialized Tasks,” Journal of Applied 
Psychology, Vol 4, Nos. 2 and 3, pp. 186-201. 
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manual clerical workers than the ordinary civil service examinations 
given these workers. : 

Dr. Otis ¢ has devised a clerical test which he reports as being val- 
uable in predicting success of clerical workers in a large silk mill. He 
also reports “zero” correlations between a composite score of “intel- 
ligence” tests and performance on the job of manual workers in the 
same silk mill. He thus concludes that intelligence is‘not an impor- 
tant consideration in success in such jobs, vA number of attenuating 
factors enter into his results. 

With all the above developments in the methods of measuring 
human fitness for jobs, more time should be devoted in the future 
to methods of measuring jobs. The lines of development in job 
analysis will be towards: 

1. Securing greater definitness of descriptions, as measures of the 
job which can be verified by other competent persons. 

2. Determining methods of comparing the relative merits of one 
job with another, with the practical point of view of eliminating the 
discrepancies in wages and working conditions of jobs which require 
approximately the same composite of talents in their workers. 

3. Rating jobs on scales of physiological, psychological, and socio- 
logical qualifications of workers. 

4. Providing for a series of gradations of preferences for jobs by 
workers on the grounds of monotony, cleanliness, and all those job 
conditions which affect the emotional life of the worker. 

5. Providing for formal methods of training talent for the job 
among the workers, and of rewarding periodically and automatically 
any acquisition of better qualifications for the job by job promotion, 
either in wages, responsibility or social recognition. 


JOB ANALYSIS 


The term “job analysis” is used rather loosely. By it is meant, as 
we have more or less implied above, the entire system of industrial 
and personnel research, sometimes referred to as a “labor audit.” In 
its narrowest sense, job analysis, made primarily for the purpose of 
hiring and training, transferring or promoting employees, is the 
analytical process of acquiring industrial facts. This is to be care- 
fully distinguished from the products of the analytical process, such 


4 Otis, A. S., “The Selection of Mill Workers with Mental Tests,” Journal of A pplied 
Psychology, Vol 4, No. 4, pp. 339-341. 
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as job specifications and personnel specifications, which are written 
reports of the work done by such analytical methods.! 

Job specifications are formulated groups of facts which relate to 
the nature and conditions of the work, the duties of the worker and 
certain conditions of service, such as pay, hours and promotions. 
The personnel specifications present the same type of facts for the 
worker, for example, his qualifications, allowable handicaps, and 
required training. Only as an abstract logical analysis are the per- 
sonnel specifications to be thought of independently of the job 
specifications. Both must be developed at the same time; both are 
logically used together in an attempt to fit the job and the worker 
to each other. 

Every job description should contain at least a minimum amount 
of description of the various operations, tools, problems and the 
like, expected of the worker in the job. In addition, a condensed 
personnel classification should be included if there are any special 
requirements on the part of the worker which he himself can easily 
and objectively measure to see if he fits up with the demands of the 
job. Such, for instance, is the amount of education required. 

As a part of the job specifications, whether or not included in the 
final typewritten or printed job specifications, or whether given on 
an independent new worker initiation sheet, there should be included 
all the information which the new applicant will want to have about 
the job outside of the duties of the job itself: the minimum and 
maximum wage to be expected, the piece work rates, the average 
length of time required by workers of average intelligence to learn 
the job, the average length of time spent at the job before promo- 
tion, the working hours, statements in regard to holidays and vaca- 
tions, and social, educational, industrial or other privileges which 
may be granted the worker. 

There should be collected other items which have a bearing upon 
the applicant’s contentedness while on the job, such, for instance, as 
whether or not the worker is required to sit or stand most of the 
time on the job, whether the work is dusty, hot, damp or otherwise 
disagreeable, by what type of men the job is manned, referring to 
such things as nationality and color. 

1 An excellent summary of all job analysis work to date is given in Meine, F., Job 
Specifications, Federal Board for Vocational Education, Employment Management 


Series, No. 3, Bulletin 45, 1919. See also Tead, Ordway, The Labor Audit, Employ- 
ment Management Series No. 8, Bulletin 43, 1920. 
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In the personnel specifications should be included the degree of 
intelligence and the amount of education, experience or technical 
knowledge required on the part of the applicant, and factors regard- 
ing the work itself, translated into human qualities, such as the 
height, the amount of strength, the quality of sight and hearing, and 
other measurable physical characteristics that may be required on 
the part of the applicant. 

The utmost care should be taken to see that whatever goes into 
the job and personnel specifications, should consist of items, which 
can be measured with some reasonable degree of accuracy by the 
people who will have to use them. Too frequently have there been 
long, wordy descriptions of the job as being one which required 
“speed, endurance, industry, skillful manipulation, etc., etc.” The 
analysis can thus be worded in such skillful manner that even a 
sophisticated job analysis expert will be misled by the fluency of the 
phrases. The principle always to be followed is simple: Whenever 
tempted to use a term like “skillful manipulation,” the job analyst 
should always ask himself, “How much skillful. manipulation is 
necessary”; and “Can the person who is to use the job specifications 
in any way measure this same trait.” 

The following questionnaire, only a beginning in this field rather 
than a completed product, will suggest some of the minimum items 
of information which it may be worth while to collect in the machine- 
like trades. This questionnaire is designed for a preliminary general 
job analysis with the expectation of later following it with special- 
ized surveys of sanitation, training and psychological investigation. 
Note that as many of the items as possible are in the form of specific 
answers to specific questions for underlining and that space is always 
left for necessary unforseen classifications. Such items as have no 
claim to objectivity may be generally recognized by having lines in 
place of descriptive answers to be underscored. Notice also the 
emphasis upon learning just where and how all available records 
of the job and workers are kept. 


GENERAL JOB ANALYSIS QUESTIONNAIRE SHEET 


Present 
INamesofn lira de sean some oi carn ee eee ioe erie Syinbo lees seahy eerie were GCodéieer as ee 


Reported  bysnemacee rane Dateate tec Weéptenee eae Woremancas sce ee 
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eb e TOR: 
A. Analysis of the Job in the scheme of: 


D. Physical Conditions of the Job: 


I. 


Place where work 


is done: 


. Posture: 


. Activity: 


Physical Condi- 


tions: 


Lighting: 


Cutting or Lubri- 
cating fluids: 


Feeds: 


MACHINE DESK FLOOR 


STANDING SITTING STOOPING 
CLIMBING WALKING LIFTING 
PULLING SHOVING yates ate: 


WALKING LIFTING PULLING 
SHOVING CLIMBING KICKING 
FINGER-MOVEMENT 
VIRUS PEN CO MABIMUDININ 6 Go Sono oacodamens 


WET DRY DUSTY 
OlbYahumessoleeee. tela VAPOR. 
LIVE-STEAM HOT COLD 
NOISY DIRTY CMEC 
CLAMMY EPECT RICA bine ae 
SOU Cent cat ieee Wocationtn tt eae 
Intensity....... Glarenneeris Shadowsaeecre 
NVikiater/ sesame race Wihentused? ecco 


MACHINE-FEED HAND-FEED  SEMI- 
HU US ROMY Dod [pt Os Siete eee th olen eiiey cure ree ere are 


a. Is speed independent of the operator? YES NO 
b. What is mechanical maximum? (Explain)................ 
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8. Rythm: REPETITIVE NON-REPETITIVE 
a. Is raw material uniform? YES NO _ (Explain)........ 
b. Is each operation uniform? YES NO (Explain)....... 
Q;, IMonotom yeti cinyein ice cite sealer ane ence ate eee ee et 


10. Motion Study (chronological sequence of operations. Describe 
briefly.) yc.aecy feces antucite Soueeccin eae a ee ak ee a on ena be eee ica 


11. Fatigue: 


b 

G: 

d. Amount of lifting done....... Weight of piece handled...... 

e. Estimated number of footpounds of work done per day...... 
12. Members used: 


a. BOTH HANDS LEFT HAND RIGHT HAND 
b. BOTH LEGS LEFT LEG RIGHT LEG 


E. Miscellaneous Conditions of Job: 
1. Does worker have to supply any tools, clothing, etc? YES NO 


4. 

5 llolidavyess POUR GRE G Ul Rie ere re ee. eee eee 
GmeNight works@Amount eae ere Ratearacar Hours worked...... 
7a OVertinie ee AMOUNT erie ei RACE SM aacth suacins to eee ergs 
8 
2) 


12. Tardiness: Amt........ Records of, (where and how kept?)...... 
13. Absences: Amt........Records of, (where and how kept?)...... 


14. 


15. 
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Are workers allowed to talk to each other during work? (describe): 
Do mechanical factors of job preclude possibility of same production 
with shorter hours? Foreman’s opinion..... Interviewer’s opinion 


F. Accident and Health Hazards: 


ToRPACCIG ENE hSicn NALOreranGcatises) nt quer ann cee 
2. Percentage of accidents to workers: (Foreman’s opinion)......... 
BW OALeL yNCeVICeSyAN GGUSe sae seus cna inward tnt: ete earparare cuneate ee 
4. Attention to job required to prevent accident:................... 
5. Liability to accident through carelessness of others:.............. 
Gaaticalin hazardsa(naturerandscalises) saree nnn eet eee ieee 
VME OCCH PAGIONAtGISCASES OMe rl Mieercrt ye Ney Noreen aati acer niees fuces Reo 
G. Inspection: 
1. How often is product inspected?........ IB Ne WON oo ong oo a0 SK 
2. Type of inspection: LIMIT-GAGE MICROMETER............ 
an Responsipility,tor inspection: ac aii ee ne nee ae ree 
4. Are finished products turned back on account of inspection? 
(WIBTELT Sed Oy) Ok Foe, Pe ae Er A oils 9 eter eo ier RIE raat: Coco 
5. How often are machine and tools inspected?.................... 
BY, NOMI RE ei etctae Ie FSA ec a ie troeer ote ee ras ats eke Mt Ree 
6. Spoiled work: Amount..n25. «.2- GOStR ay Betti chseeaaunele eee te eres 
Carises EVO NEAN SMEAC HINES shrgetea tere octane eke seiner 
7. Are records of spoiled work kept? YES NO (Describe)........ 
8. Can spoiled work be traced to the operator producing it? YES NO 


MDD ESEHIDE) receipe oars ais hele OR ec eake) ivr eae Re eeecoote Nilay s aioe 


H. Production, Turnover, Security of Job: 


3. 


4. 


Production: 

a. Average production of workers on job: (Foreman’s estimate) 
Dee Mininiinete en eee Maximum:. otic eniaente 
Foreman’s opinion as to whether greater orédartion Aesibte Sai 

GESITAD IC RATER ET sone ec HO oo erste. ave ain tsbe travers 
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7 “Rush orders: (nature) 2p iiurdcncipecaee Oro Oe ee eee ee 
8. a. Is worker idle while setting up machinery, tool setting, in- 
spection, repairsrs, (Describe) aa semen eee eee 

b. Does he rest in such intervals? (Describe)................ 


g. Labor turnover on this job (Foreman’s estimate):................ 
a. Average length of employment of worker on this job. (Fore- 
Man S- Estimate). coxce lies ene ee eee ae ee eee 

b. If men and women are both employed, which stay longer? 
(Poreman’srestumate)s so 4c eee ee ee ee 


10. a. Is production adaptable to count YES NO _ (Describe).... 


11. Average length of training for average worker to become profi- 
Gent, Onsfobreten, tise tenor nora Meson eee 
12. Costof labor. turnovers.cn ee een ee eee ee 


I. Labor Supply: 
1. Any apprentices (understudies) being trained?................... 
2. Deficiencies in qualifications of workers (Foreman):.............. 


3. Will supply meet demand in future? (Foreman) YES NO 


(DDSSCrIbe) s4hcis.5 48 cos eae TOR, ieee ee ne 
Av SOULCE OL SUp ply carts weaved costo ae eco eC ee 
5. Transfers: Amount........ Per cent formed of turnover.......... 
6. How are workers selected (function of foreman)................. 


II. THE WORKER: 


J. Description of Duties: 
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Special duties (if sub-group of larger group, e.g., dial press tool setter 


Miscellaneous: 


AE SCEALOO See nee Mert ele i Se 


= 


No} ed oS) ey a oe eS) 
v3) 
(q’) 
ue) 
oe 
ay 
5 
eI 


Lol 
iS) 
w 
oO 
n 
>} 
fe) 
5 
a 
‘s 
= 
<< 


K. Qualifications: 
Previous trade training required: 


ites I BIRUYSyy mY SEEKS cones, arse Meegencege DLA WInleSheeatee ene thon cae 


num fw N 
zy 
© 
3 
ig’) 
o 
° 
Lona) 
ct 
° 
S 
D 
3 
© 
ie) 
ee 
3 
@ 
at 
uo) 
© 
ta 
eh 
o 


BU WfeveXel evo heaes Ban cia Pocateles Conteh ach uci fetes ee a ie ie ticy EOCENE REM ees en 
Fis AGG Distel flojlShe ar aden alae See IS OS CHa Rae te eR eR A 


Requirements of schooling, intelligence, training on the job. 


i AS EIS TONS SUE DO. WING ORE DID ein an eco dern.anotocue oo 
2S CH OOWMUN Gee ae hae na cet arent eats acto. cite Moore rd eee 
SB eealechnicalktralnin over, ces ase tke niche aes euekecoendtors fae ious mim oeeeT a see 
4. Deficiencies in TRAINING of workers (Interviewer):............ 
5. Necessary previous experience in same or allied job:............. 
Ss Pte Aro he Si SRR RT aM NASON Te ar eee ae NV Wats |Obeecwie me eng 

6. Average length of time for average green hand to come up to average 
pleceatecarmingsenkinse ys tic cer denne se net eee astiat lec casein 

7. Are motions, methods, feeds standardized?...................... 
8. Qualifications for a worker to be trained (reasons):............... 
Omi tite licence sewer ea tered eee aie Atel siete casts sia aad piace neta neonegs 
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II. 
12. 


iG. 


14. 


15. 
16. 
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How introducedintoimmewsj0b hare ace se ieee nist ei erent 
Is new worker followed up in ten days?. YES NO............. 
Any written or oral instructions to new employee? YES NO 


Required ability in mathematics: COPY FIGURES, ADD, SUB- 
TRACT, MULTIPLY, DIVIDE, DECIMALS, SOUARE 
ROOT, SOLVE FORMULAE, CALCULUS. 


Hiteracy3 sacs lV- ORDERS: READ ORDERS 
UNDERSTAND-VERBAL-ORDERSG....... 


Physical: 


20. 
21. 


. Eyesight: acuity...... glasses an objection...... Colors) ee eee 
ww(F@atino: 2 Va Cutty, syate ces. syne Aicaaecchorei tio eens ee role IU nae 
wyexks MP? (Reasons) anc .oe: aide con een ee ee ee 


Wear fingerstalls? YES NO 
General strengthi!:0.5.425 cnr a, bono) See coe ie one eee 


Can mental defective do the job? YESS NOM Remarks:ee nese 
Can left-handed person do the job? YES NO Remarks:...... 


L. Wages and Promotion: 


I. 


2. 


3. 


Do workers:know dine of promotiont enn else ate eee 
Doypromotionsicomesunaskeditontnmimeeis ct eerie eee 
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4. Any definite plan of promotion? 
5. Wages: 


Basis of Pay 
Lowest Highest Average: 
Day rate 


Piece: rateme mie Pan ees) ae 


Continsentirate ss) 9... 


6. Nature of contingent rate: 


So28;8) [0s] (s 0901S) pie @) es) eke! bea csi@lerels « «) 6: (el eile #) 0] \s-s) 61 ug 


15. Are wages dependent upon: WORKER’S-OWN-ABILITY 
GROUP-SBER Dime erate rere c ince tartromnn ca c e mete eto N ee ee tor at te 


It will be noted that much information is of the purely subjective 
type, as, for instance, “attention required, degree of fatigue, etc., 
etc.” At the present time but little better can be done than these 
subjective judgments which approach dangerously close to the type 
of character analysis which we have criticized so severely. It is desir- 
able to find objective methods for expressing these items of a job. It 
would seem possible to devise a very simple objective rating scale 
for monotony, for instance: 


DEGREE DEFINITION 


I as monotonous as a dial press job. 
2 as monotonous as a production lathe job. 
3 as monotonous as a criminal lawyer’s job. 
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Such a scale with intermediate degrees,-carefully and thought- 
fully worked out, would yield results by means of which various jobs 
might be semi-objectively compared for monotony. Outside of being 
valuable in turnover investigations, such information is of little or no 
value in hiring, since no one is competent and no test method is 
available which will predict with surety in advance whether an appli- 
cant will work contentedly or not at monotonous work. If, however, 
an applicant should report to the interviewing examiner that he 
had worked steadily for X years at a job Y, which the examiner 
knows is a job with degree of monotony 1%, then he would have 
some assurance that the applicant would be satisfied on a similar 
job Z with the same degree of monotony. Men do not fall into two 
clearly defined groups, the honest or the dishonest, but are of many 
degrees of honesty; jobs likewise are not monotonous or varied, but 
of all degrees of monotony. Furthermore, the job analyst must not 
fail to make the subtle distinction that a job, monotonous to himself 
to the extent 7, may have a lesser monotony to the worker who 
works at that job as represented by the expression, 7 — K. 

There is unlimited room for improvement in the methods of 
measuring jobs. This statement does not imply that there is not 
also room for improvement of tests used in measuring human capaci- 
ties. One line of development, as yet but little used, is that of meas- 
uring by standardized tests a worker’s performance on the job. In 
the training school of industry this aspect of human measurement 
would take the line of measurement of product turned out after a 
given amount of trade instruction on standard test jobs of produc- 
tion; in the trade school, it will mean performance tests, of a stand- 
ardized nature, given to the students in trade courses to measure 
their ability in the course taken; while in the vocational guidance 
bureau, it will mean subjecting boys and girls to trade instruction 
with the sole purpose of testing their ability to learn the trade proc- 
esses taught. It has been the privilege of the author to become 
acquainted with an educational experiment which combines both 
the latter purposes in a “trade extension school for vocational gui- 
dance.” 

EVALUATION OF VOCATIONAL GUIDANCE TESTS 


At the Manhattan Trade School for Girls, New York City, there is 
being conducted one of the most interesting experiments to be found 
anywhere in this country in extension education and vocational 


r 
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guidance by means of tests. The uniqueness of the plan, thé agmibers 
of pupils given advice, and the fact that tests are being used actually 
in basing action upon them, rather than as mere interesting but 
unused researches, entitles this extension department to an enviable 
position among the foremost vocational guidance programs to be 
found to-day. Every second week throughout the school year, a new 
group of from 100 to 120 eighth grade public school girls from various 
public schools of New York City come to this extension department 
for a two-weeks course. The aim of the department is both to in-— 
struct the pupils on various vocational opportunities and to test ° 
by means of performance tests their ability to learn a few funda- 
mental things about a limited number of these trades or,;vocations. 

The teachers of this extension department make occasional visits 
to the industries, working on follow-up reports of girls placed in 
industry by the school, and thus become acquainted with the facts 
of the various industries touched upon in the extension school course. 
Thus information becomes available in regard to average wages, 
range of wags, trade opportuaities, active and dull seasons sani- 
tary and other working conditions of the various trades. These facts 
are imparted tothe girls by the respective commercial and indus- 
trial teachers he information given on commercial work covers 
information of the many subdivisions of commercial work; for in- 
stance, the positions of typist, stenographer, secretary, telephone 
operator, computing machine operator, filing clerk, bookkeeper, etc. ~ 

In addition to this work, educative and highly desirable as a a 
of the vocational guidance program, the larger y par e two weeks 
time is given over to the performance tests>~~ ie Seer plan ‘of the 
tests is first to give oral instructions in the method of doing a funda- 
mental operation, and then to judge unis ied the excellence of the 
objective product produced. 

The pupils are rated by the fe cieres + approximately the end of 
every hour’s work during the fifty hours spent in this extension 
school. In some cases it is possible to secure independent ratings of 
two different teachers of the same industrial subject, hand sewing, 
for instance. / This makes for more reliable ratings when the double 
ratings are possible. The great amount of rating required neces- 
sitates that the teacher become acquainted as rapidly as possible with 
the pupils. What might to the outside observer appear as an exces- 
sive amount of rating is therefore very valuable in insuring that 
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judgments will be made in as few cases as possible without knowing 
pupils. The large amount of rating required-means that the time 
that could be devoted to objective scoring of products is very limited; 
in fact, to the extent that at the present time no objective scoring is 
done. Objective scoring, using scales, tests scored by stencil or 
otherwise, could be adopted only by devoting to such work time now 
spent in subjective scoring. 

As a final record of each pupil, a qualification card is made out in 
duplicate. One copy of this card is retained in the files of the exten- 
sion school for possible later reference or follow-up work; the other 
is sent to the principal of the school from which the pupils came, to 
to be used by him in supplementing his own guidance given to grad- 
uating pupils. An evaluation of the items of this qualification card 
is here undertaken in an effort to determine the value of the several 
tests, both theoretically and practically, with the hope of suggesting 
improvements in the present tests or test technique, with the prac- 
tical point in view of securing (a) better predictions of after-school 
sucess in the vocations undertaken, and (b) greater ease and objec- 
tiveness of scoring the various tests in reporting the results. 

The scores on the qualification card are not the scores received on 
the several tests; the hand sewing scores of the qualification cards, 
for instance, represent the results of the first subjective summation 
of scores in the four abilities of hand sewing, “accuracy, speed, neat- 
ness, handling,” in making a simple hem handkerchief and a child’s 
cap; the pasting scores of the qualification card represent the first 
subjective summation of a series of scores of the four abilities of 
pasting “accuracy, speed, neatness, handling” in doing sample 
mounting of colored cloth triangles upon a cardboard backing and 
making a paper covered pasteboard box with cloth reinforced 
corners. 

The test scores of qualification cards are subdivided into com- 
mercial tests, ten in number, and industrial tests, nine in number. 
To evaluate these, a celluloid stencil was constructed, on which each 
test and each score or rating on a given test was arbitrarily given a 
numerical value. It is only by assigning numerical values to “high 
average” position of check mark on a line opposite the English test, 
that such scores can be effectively dealt with statistically. 

The commercial tests scores were all reported as “good, fair,” and 
“poor,” finer differentiations of ability being secured by significance 
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being attached to the position of the check mark, indicating the 
score in the test, upon a ruled line approximately 2.6 inches in length. 
On the celluloid scoring stencil, this length was subdivided into nine 
equal parts, numbered from I to 9, in increasing order of merit of 
score obtained. The position of the check mark as falling in one of 
these nine vertical columns of the stencil is the score in each of the 
commercial tests, the variables being numbered 5-14, inclusive. 
These check mark estimates were subjectively evaluated and a com- 
mercial “stamp,” or recommendation of the director, placed on the 
card as variable No. 15. The various stamps and their evaluation, as 
well as the similar industrial “stamps,” are explained below. The 
industrial tests, in their various subdivisions, had their scores 
reported as “good, fair” and “poor,” arbitrarily given numerical 
values respectively of 3, 2, and 1, in variables numbered 16 to 34, 
inclusive. 
The several tests, briefly described, are as follows: 

I. Commercial Tests. 


Variable 5. English. Three compositions and a final test are required in the 
course: 


A. Composition: Skilled and Unskilled Labor. 
B. Composition: Vocations for Women. 
C. Composition: Effect of the Introduction of Machinerv. 
D. Final test. 
a. Dictation: Discourse between two people. 
b. Composition: What These Courses in the Extension Classes have 
Meant to Me. 
Time: 


Lectures, preliminary to compositions, 30 minutes each. 
Compositions, 30 minutes each. 
Final test, 60 minutes. 


Variable 6. Arithmetic. 
Single problems in arithmetical operations. 


Variable 7. Penmanship. 
Based on penmanship on written work of compositions. 


Variable 8. Switchboard. 

A series of demonstration performance lessons on the use of a private branch 
exchange switchboard, the various pupils taking turns in being, “private 
branch operator,” “lady in a residence,” “desk clerk,” etc., being rated by the 
teacher in each capacity in turn. 


Variable 9. Filing. 
A series of three lessons with tests on each. 
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A. Alphabetical filing. Test of arranging 50 cards in alphabetical order and 
copying the order on paper. 

B. Geographical filing. Test on same. 

C. Telephone Directory and its use. Test of using directory in locating desig- 
nated names. 


‘ 


Variable 10. Stenograpby. 
A series of lessons, beginning with the single strokes and leading up to the final 
construction of simple words and sentences. Ability scored on the basis of 
reactions to simple dictation involving beginning principles of stenography. 


Variable 11. Typewriting. 
A series of five lessons leading up to the final formation of simple letter groups, 
or syllables, by the touch system. Practice sheets are preserved and sub- 
jectively scored by the teacher. 


Variable 12. General Adaptability. 
A subjective judgment of presumable general intelligence. 


Variable 13. Personality. 
A subjective judgment. 


Variable 14. Appearance. 
A subjective judgment. Stressing neatness rather than personal beauty. 


Variable 15. Commercial Estimate, or “stamp.” 

This is the director’s subjective summation of all the check marks of the above 
tests. This is the recommendation of action to the principal of the school from 
which the pupil comes. A series of rubber stamps are used, an imprint of one 
of which is affixed to each card. As arbitrarily evaluated, after comparison of 
meanings of the various ratings of the various teachers by drawing on a ten- 
inch line the division points of the various meanings, the stamps were given 
numerical scores ranging in ascending order from I to 7. 


II. Industrial Tests 


Variable 16. Teacher’s subjective evaluation of hand sewing ability, based on the 
card entries of the four following sewing variables; accuracy, speed, neatness 
and handling, variables 17, 18, 19, 20. For the 97 persons of the experiment 
this correlates with the sum of the gross scores of the four variables, of which it 
is a subjective summation, to the extent of .662. There was a fair spread of the 
summation scores. 


Variable 21. Teacher's subjective evaluation of power machine operating, based 
on the card entries of the four following power machine sewing variables; 
accuracy, speed, neatness, and handling, variables 22, 23, 24, 25. For the 97 
this correlates with the sum of the gross scores of the variables, of which it is a 
subjective summation, to the extent of .881. 


Variable 26. Teacher’s subjective evaluation of pasting, based on the card entries 
of the four following pasting variables; accuracy, speed, neatness, handling, 
variables 27, 28, 29, 30. For the 97 this correlates with the sum of the gross 
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scores of the four variables, of which it is a subjective summation, to the extent 
of .868. 


Variable 31. Grasp of Directions. 
A subjective judgment, presumably “ability to imitate” demonstrated _per- 
formance. 


Variable 32. Persistence. 
A subjective judgment. 


Variable 33. Interest. 
A subjective judgment. 
Variable 34. Originality. 
A subjective judgment, “designing” ability presumably predominating. 
Variable 35. Sewing estimate, or “stamp.” 
The stamps were given numerical scores in ascending order of merit from 
0 to 4. The power machine and hand sewing evaluations of variables 16 and 
21 are here subjectively combined in the one stamp, making impossible sep- 
arate evaluation of these two items. 


Variable 36. Pasting estimate or “stamp.” 
The stamps were given numerical scores in ascending order of merit from 0 to 4. 


The above descriptions are but meager and inadequate repre- 
sentations of the tests used and the numbers of ratings made. This 
research is devoted to the evaluation of the qualification card entries, 
and not to the original ratings from which these are made. As such, 
the results reported logically fall into two headings: 

A. Intercorrelations of various additional tests administered to 
97 of one such section of pupils, on whom qualification card records 
were complete, together with similar intercorrelations of the quali- 
fication card data and test scores. 

B. Evaluation of the qualification card test scores against aver- 
age semester school marks in commercial high school subjects of 30 
pupils who had passed through this extension school previously, and 
were found to be attending commercial high school at the time of the 
investigation. 


A. Results of Intercorrelattons Based on 97 Pupils 


With the aim of investigating the possibilities of standard objec- 
tive tests in conjunction with the subjective performance tests, addi- 
tional verbal and non-verbal intelligence tests were given to one of 
the bi-weekly classes of the extension department. There were 97 
girls who had complete test records in all tests. 
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In the results which follow, it is to be remembered that the reli- 
ability of the gross scores is somewhat greater than is ordinarily the 
case with subjective judgments, since each gross score of the qualifi- 
cation card is normally the result of a subjective summation of 
several personal judgments, made in some cases by two or more 
teachers independently. 

In order to evaluate the qualification card variables against one 
another, combinations of the scores of the variables were made by 
adding the gross scores and computing the intercorrelations. These 
intercorrelations are shown in Table IV. 

Of the commercial tests, switchboard, stenography and type- 
writing (8 + 10 + 11) combined have the highest correlation with 
the director’s estimate (15), 7 = .676. Next in value in predicting 
the estimate (15) are English and arithmetic combined (5 + 6), 
ry = .595; next, general adaptability, personality and appearance 
combined (12°+ 13 + 14),7 = .575; and finally penmanship and 
filing (7 + 9),7 = .506. In view of the magnitude of these correla- 
tions, the average of which is .588, it would seem that too much of 
personal judgment of the director is being injected into the final 
estimates, over and beyond what the tests themselves show. The 
weighting of these variables in determining the final “stamp” applied 
to the card is a subjective matter. The correlations wouid lead us to 
believe that much might be gained from using a series of objective 
weights in determining what is to be the final stamp. 

The majority of the industrial tests, as combined, correlate nega- 
tively with the commercial estimate. The average of the nine com- 
binations of correlations of industrial tests of the table with the com- 
mercial estimate is — .003. The final commercial stamp (15) corre- 
lates with sewing estimate to the extent of — .118; and with pasting 
to the extent of — .008. Taking the tests at their face value, one 
would conclude that commercial ability and industrial ability in sew- 
ing and pasting are correlated slightly negatively. Grasp of direc- 
tions, persistence, interest and originality in industrial work (31 +32 
+ 33 + 34) combined correlate positively with commercial estimate; 
r = .306. This suggests that these subjective tests of character 
qualities re influenced by the same considerations which make for a 
correlation of .186 with general adaptability, personality and appear- 
ance (12 + 13 + 14) of the commercial tests, and a positive corre- 
lation with all other commercial tests. These four subjective judg- 
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ments could well be discarded for a non-verbal intelligence test, for 
which they are a very inferior substitute. 

All industrial tests intercorrelate among themselves positively. 
The average of nine correlations of the table with the final sewing 
stamp gives r = .483, and with the pasting stamp, 7 = .516. 

Combining “accuracy, speed, neatness and handling” by adding 
the scores, gives lower correlations with the final industrial stamp 
than the subjective evaluation of these items; in the case of sewing 
evaluation and sewing stamp, .618 vs. .920; sewing estimate and 
pasting stamp, .543 vs. .647; power machine operating and pasting 
stamp, .718 vs. .797. In the other cases the combined score corre- 
lates higher with the final stamps; power machine operating and 
sewing stamp .474 for the estimate vs. .569 for the combined score; 
pasting evaluation with sewing stamp, .305 vs. .308, or equal so far 
as all practical considerations are concerned; and pasting evaluation 
with pasting stamp, .374 vs. .377, again equal for all practical pur- 
poses. Inasmuch as the scores in hand sewing have little dispersion, 
it seems true that subjective bias has entered into these hand sewing 
scores, and consequently into their final evaluation and is reflected 
almost perfectly from the evaluation in the sewing stamp applied. 
Hand sewing evaluation and sewing stamp correlate to the extent of 
.920. Statistically, the difficulty lies in not making use in the final 
stamp of the differentiation already secured in the tests. Judgments 
of “accuracy, speed, neatness, handling” are based on more than 
one judgment and so have some reliability in themselves. The 
evaluations can go only by full steps; the summation can add up by 
partial credits. That is, the evaluation of A’s ability must be either 
I, 2, or 3; while by summation it may be any score from 4 to 12 in- 
clusive. The correlations afforded by the evaluations of the stamps 
are such as to prove the feasibility of using the sum of the four. 

Alpha (Form 6) and three Thorndike Non-Verbal Intelligence 
Tests (Forms: I-A, I-L, II-K) were given to the 97 subjects. The 
intercorrelations are shown in Table V. It is to be noted here, as in 
the case of all correlations based on these 97 subjects, that while the 
correlations apparently are low, these correlations are based on a 
very limited range of talent, the eighth grade of school to which all 
belonged. Since all are of the same grade, presumably the older 
pupils are the less intelligent; this is brought out by the four out of 
five negative intercorrelations. The Toops-Pintner revised direc- 
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tions test was also given. This test is a poor measure of whatever 
abilities are measured by the three non-verbal tests. As a short, 
easily administered, and perfectly objective, scorable-by-stencil test 
of intelligence, the test has promise; its correlation with Alpha is 
higher than the average intercorrelations of the three non-verbal 
tests. Of the three non-verbal tests, Form I-L, which is most diffi- 
cult of the three forms, has the highest average intercorrelation. 
This seems to suggest that more difficult non-verbal tests might 
prove to be of more value than the present ones. 

The correlations of the two intelligence tests and the combined 
score in the three non-verbal tests, found by adding together the 
separate gross scores in the three non-verbal tests, are shown in 
Table VI. The striking contrast between the value of Alpha and the 
value of non-verbal tests in predicting respectively the commercial 
test scores and the industrial test scores is shown by the plus sign 
which indicates which of the two “intelligence” tests is the better 
predicter of the respective test scores. If we say that the Alpha is 


TABLE V 


INTERCORRELATION OF MEASURES OF INTELLIGENCE 
ALL EIGHTH GRADE Pupits. N = 97 


Intercorrelation With: 


Average 
ith otf 
Variable Age  |Directions ee ae poe Alpha ' ‘ ie 
Rence 
measures 
Age Maas —.242 —.038 .O51 —.IIt —.276 —.123 
Directions —.242 Soe —.019 -040 034 -398 -113 
ING WEEN —.038 —.019 Meee ‘473 -287 5227, +242 
N. V.-I-L -O51 -040 -473 ores +375 «319 .302 
N. V.-II-K —.III .034 .287 +375 eine -4II Pal 
Alpha —.276 -398 7227, +319 411 a -339 
Variable Average 13.45 15.65 43.02 | 39.45 | 45.30 | 65.80 
o 824 3-395 12.081 9.258 9.519 14.322 


Trade Tests in Education 


86 


MOYS SUIS SN{d 9Y} ‘S}S9} VATWDIdSaI oY} OJ ‘vy pue € SMOI UT 


*S9IONS $9} [PIIISNPUT IY} JO S19}0Ipeid 10730q Sosed ][e Ul sIe Poul 
-WI0D $}S9} [EGIOA-UOU JaIY} IY} I[IYM ‘SoIOS 4S9} [EIOIOUTUIOD []e Jo 19}Ipoid 109330q *e SI eYdTY ‘UOTyds90x9 NOY “eJOIpeid 193}0q 9Y} SI 
YOIYA 4S9} JeY} 0} UdAIT Suleq usIS sny{d 9y} ‘sa1Os 489} U9ATS OY} JO 19}dIpaid 10330q & OIe S}S9} [eqIaA-UOU PosUIqUIOD 9y} IO eYdlY JoyWYyM 
‘Seb’ = sysay [eqiaA-uON peurquiog seiy, pue eyd[y Jo uoreeri0g 


ae ik ae ae Be a a + ay) |) paceed|sccte |) Godocd(sooece.|! edie [eq12A-UON pourquiog samy 
a ePESE Og) eee ee. were ese ake ea biel fehl wae ee iin 8) US) Wee 6 SUS RN Gl AEP eke Oe ei .6.w womne + + + + + eud[y 
Sz Lvz: zee" eve: vot: | goz vou 8gz* gcz goz Sr v6z vee: Ivr’ [eqiaA-UON pouIquIOD sx214. L 
vro'— | fzo-— VLI for fer zgo'— | €11°— | SEo°— 900° 1ov vce: S6f vsr 6Le: eydiy 
x 4 [eywQO| Gy | BV ISS BS vss m] Sy a lewals2ual] wv ty 
3 SY x 3 za 
S| € (RS e( S82) 8S (ee sss Sees) sei s sss ss § 1s 
SS LAS eee SS ee eS Si Se = 
o | @ eel S 1 Reema Se) ee | a lB) 88) elo 
s s jes“) 8 Se |sFsisse| FS] Fs Sy spl ag] = : 
3 = | Sp] es [Ss S388 Sales) = aS Sob OS ie 
s SD TRS =} BSS Sess Selo cS el Se Ih Bes 2 pe] 3 e = 
~F} is Selcece.s = 8 BS) & 8 Sell SSA oS, SS 
SS | SS =. SS aa | =o 3 Se SS = 
aos Ses S) > ie) o & Ses s gs &§ 3 
ss| 8 = ns s 8 =| x oO Se Se SS = & 
mS eas S Sas g aS ko aS ERS Sa Sy = 
25% = = Sons g = = ; a = 
' 


[YnM UorDa440D 


46= N “IVIMLSOGN] GNY TVIOUAWWOD 
‘SISH], SQOTUVA HLIM SISA] IV€YAA-NON 4O SAYOOS GANIGWOD AAAH] JO WAS GNV VHdTY dO NOILVTEXAOD 


IA ATaVL 


Tests of Trade Capacity or Promise 87 


“typical” of commercial test scores and the non-verbal test of indus- 
trial test scores, and if into the subjective evaluation of the four 
“abilities” in each of the three industrial tests of hand sewing, power 
machine operating, and pasting there enter largely those personal 
subjective qualities otherwise designated as “personality,” “general 
adaptability,” etc,—then we should expect to find the following: 

1. Alpha correlating higher with the subjective evaluation of the 
four abilities than the objective arithmetical summation of these 
four abilities. This it does (Table VI) in two cases out of three, the 
exception being in power machine operating, in which it may be 
noted, the test product is more easily scored objectively than the 
hand sewing or pasting products. 

2. Non-verbal tests correlating higher with the objective arith- 
metical summation of the four abilities than with the subjective 
evaluation. Thisthecombined non-verbal score does in all three cases. 

The argument thus seems strongly in favor of scoring the indus- 
trial tests solely on the basis of the objective tests, and not allowing 
for a subjective judgment “over and beyond” what the objective 
arithmetical summation of the tests alone would give. 


B. Follow-up of Students who Entered Commercial High School 

From the records of the extension school was found a list of 134 
students who had signified their intention of entering the commercial 
course in Washington Irving High School. From this list only thirty- 
nine records could be located at the high school (29.1 per cent). This 
fact is mentioned as showing one of the difficulties encountered in 
checking up the value of any selection methods. Undoubtedly more 
than thirty-nine students had entered high school, but had dropped 
out, since there were a few of the persons who were at the time in 
their fifth semester at the high school. Our records therefore are 
lacking in one important element,—knowing the test record and high 
school record of previous failures. From the thirty-nine records, 
there were available only the thirty records sufficiently complete in 
all respects to be used in partial correlation. 

The thirty-nine students had been in school for the following 
lengths of time: I semester, 0; 2 semesters, 2; 3 semesters, 4; 4 sem- 
esters, 31; 5 semesters, 2; total, 39. 

The average per semester percentage school marks were com- 
puted in the following subjects: English; mathematics; drawing; 
stenography; typing; bookeeping. 
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A few unmentioned electives were disregarded in obtaining the 
average per semester percentage school mark in all subjects. This 
average school mark is inaccurate through the rating of different per- 
sons by different teachers, through changes in the standard of grading 
from year to year, through unequal lengths of time in school and con- 
sequent different numbers of semester courses, together with all the 
errors inherent in a percentage system of grading. It is a rough crite- 
rion of ability to do commercial high school work, but is the best one 
may obtain without excessive statistical labor spent on only a few 
cases. Criteria for evaluating test methods are generally subject to 
such difficulties. 

A check on the value of the extension school tests in predicting the 
probable commercial high school success of these pupils is the corre- 
lation between their extension school tests and commercial high 
school grades in the same or similar work. Table VII shows the 
results. 

It is seen that the typing and stenography tests, approaches to an 
objective trade test method, are fair predicters of high school marks 
in those subjects. Extension department arithmetic tests are fair 
predicters of bookkeeping marks, but not of the marks in algebra, 


TABLE VII 


TABLE OF CORRELATIONS OF EXTENSION SCHOOL TESTS WITH SAME OR SIMILAR 
ScHooL CouRSE AVERAGE PER SEMESTER HicH ScHOoL Marks 


~ 


Correlation of Number of Cases r= 


School Mark in Typing with Typewriting 
Test Score 24 .430 


School Mark in English with English Test 
Score } 30 .205 


School Mark in Stenography with Stenography 
Test Score 23 -452 


School Mark in Mathematics (Algebra) with 
Arithmetic Test Score 14 +.135 


School Mark in Bookkeeping with Arithmetic 
Test Score 26 .478 
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a more abstract mathematical subject. The English test is a poor 
predicter of high school marks in English. Probably all these corre- 
lations, seemingly not very high, would be higher if the high school 
group were not of so restricted arrange. Furthermore, the group 
studied consisted of only the “survivors” and not the “entrants.” 
Again, the extension school estimate must be one which will predict 
the average success in a subject for as long period of time as five 
semesters in high school. The results reported above in the case 
of the English test, a composition test, which in common with most 
composition tests probably has a high P. E. of the individual score, 
suggests the need for a revision of this test. 

The relationships of the high school commercial course marks to 
the various “stamps” in commercial tests, sewing tests, pasting tests, 
are shown in Table VIII. Commercial “stamps” correlate positively 
to the extent of .403 with commercial high school grades. It is this 
figure which we hope later to raise by the partial correlation weight- 
ing of the various commercial tests. High school marks correlate 
positively with sewing “stamps” to the extent of .242; and with past- 
ing “stamps” to the extent of .063. There is evidently little relation- 
ship between ability to get along well in a commercial high school, 
as measured, and ability in sewing or pasting, as measured by the 


TABLE VIII 


INTERCORRELATIONS OF “STAMPS” IN COMMERCIAL, SEWING AND PASTING TESTS 
WITH AVERAGE PER SEMESTER HicH ScHooL Marks. N=30 


Correlation with Variable: 


Variable 
I 2 3 4 
I 1.000 403, .242 063, 
2 403 1.000 .346 336 
3 242 -346 1.000 597 
4 063 336 597 1.000 
o 6.320 1.585 .829 +539 
Average 67.20 3.47 2.67 3.10 


Variable 1—Average per semester school mark in commercial high school courses. 
Variable 2—Estimate “stamp” of commercial ability. 

Variable 3—Estimate “stamp” of sewing ability. 

Variable 4—Estimate “stamp” of pasting ability. 
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extension school “stamps.” The fact that, where we have objective 
evidence in regard to the pupils’ commercial abilities, the correla- 
tions of commercial success with the industrial stamps are positive in 
every case, would lead us to doubt whether the low negative correla- 
tions of Table IV really indicate a fundamental negative relation- 
ship between “commercial ability” and “industrial ability,” or are only 
the result of the teachers’ possible adherence to a “type” philosophy 
of tests,—the ever popular belief that a person “preeminently” fitted 
to be a commercial worker will not be a good industrial worker and 
vice versa. Such a chance for bias, not possible with objective trade 
tests, may impose a very real obstacle to the success of the subjective 
method of judging the results of objective tests. 

A previous tabulation of the commercial “stamps” and subjective 
composite of the sewing and pasting “stamps,” called “industrial test 
rating” in the case of 346 successively accumulating qualification 
cards, yielded a positive correlation between commercial “stamps” 
and industrial “stamps” of .o71. Commercial ability, as measured by 
these subjective stamp evaluations of commercial tests, is related but 
slightly to industrial ability similarly determined. 

The intercorrelations of the various commercial tests, the corre- 
lations with average per semester school marks, the average scores 
and standard deviations are shown in Table IX. 

These tests, excluding the estimate, variable 15, were evaluated 
by partial correlation yielding the formulae: 


In terms of deviations: 


English Arithmetic Penmanship Switchboard 
x a 
Ants 5 X6 x7 x8 
— = —.01I5— +.173 — +.184 —— +.036 — 
(8) oA 05 06 07 08 
oe General 
Filing Stenography Typewriting Adaptability 
Xo X10 X11 v12 
+.110— +.120 +.280 = .137 
d9 710 O11 O12 
Personality Appearance 


n x : 
—.008 —* —.047 —“ The total correlation of the 
o13 O14 


composite and criterion is, ric = .712. 
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Or, in terms of gross scores: 
(9) *%, = — .040 X, + .508 Xe + .509 X7 + .102 X38 
+ .350 X9 + .307 X10 + .943 Xu + .424 Xv 
— .034 X13 — .161 X14 + 53.238. 


We are now in a position to state the value of the several tests. 
We shall do this by variable numbers. 


Variable 5. English Test. As shown by the very low negative weighting, the 
test is, for these follow-up cases, worse than useless as now scored. As pre- 
viously shown, the English test is of almost no value in predicting high school 
marks in English. A composition test has too high a probable error of the indi- 
vidual score to be of much value as an English test. A Trabue Completion-Test 
Language Scale, requiring not more than ten minutes, would probably give much 
better measure of school English marks, and besides is a good intelligence test. 

Variable 6. Arithmetic Test. A highly valuable test, exceeded in value only by 
the penmanship and typewriting tests. 

Variable 7. Penmanship Test. Exceeded in value only by the typewriting test. 

Variable 8. Switchboard Test. Of little value in predicting high school marks in 
a commercial course. The test may prove to be of value as a vocational test for 
switchboard operators. For predicting high school commercial course success, 
the time spent on this test could well be devoted to other tests of promise, a 
good intelligence test, for instance. 

Variable 9. Filing Test. A desirable test, exceeded in value only by five others. 

Variable 10. Stenography Test. A desirable test, about on a par with the filing 
test. 

Variable 11. Typewriting Test. The most valuable test of the group. The spread 
of scores (standard deviation) is less than in any of the other objective tests: if 
a greater spread of scores could be obtained, the test might prove to be an even 
better test than at present. 

Variable 12. General Adaptability. This is a subjective judgment test and is 
about on a par with the filing and stenography tests. The trait probably cor- 
responds most nearly to general intelligence of any of the personal traits judged. 
In spite of its being a subjective test, it receives a fair positive weighting, suffi- 
cient to suggest greater possibilities from an objective intelligence test. The 
administrative demands upon such an intelligence test are: (1) Short adminis- 
tration time. (2) Short scoring time. (3) High reliability. Some form of omnibus 
intelligence test will meet these requirements. If the present English test 
should be abandoned in favor of a Trabue Language Scale, there would be less 
need of such an intelligence test. Again, the time now spent on the switchboard 
test would give more than ample time for such an intelligence test, if the switch- 
board test were abandoned. 

Variable 13. Personality. This is another personal judgment test. It receives a 
negative weighting, which might suggest that “personality” is “bad personality” 
that remains after a pupil has been judged for intelligence. The time taken for 
this useless judgment might well be spent in scoring a good intelligence test. 
The spread of scores is least of all the objective and subjective tests. 
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Variable 14. Appearance. Although the spread of scores is higher than the pre- 
vious test, it would seem that “personal appearance” is of no concern in com- 
mercial high school success! One might, of course, argue that thetrait determines 
to some extent one’s success after leaving school and entering business. This 
seems doubtful, if one may judge from popular opinion, for the homely girl is 
often preferred in many clerical positions as it is commonly believed that “her 
chances of being married are less than those of a pretty girl.” At any rate, it 
seems doubtful whether a public school should ever bar anyone from taking a 
commercial course on such grounds, 


The results of this investigation are convincing to the effect that 
objective tests of the type used can be made a very worthwhile part 
of the educational machinery involved in preparing children for life’s 
struggles. The results are promising enough to justify the army 
trade test procedure being adopted in the case of one or two hundred 
high school freshman students whose abilities will become known. It 
would be but a simple matter to have the same tests, as are now 
administered to the eighth grade girls, administered to a class of high 
school students who would attend the extension school for two weeks 
for the purpose. During this time, an additional number of intelli- 
gence tests, reading tests and other tests of promise should be tried. 
By standardization of the tests on large numbers of pupils who will 
be certain to attend the high school, we will include the “to-be-fail- 
ures” in our group as well as the “survivors.” This procedure con- 
forms to the procedure of trade tests, adapted to vocational guid- 
ance. It should be evident that testing students who have already 
had extended high school training in stenography, typewriting, and 
the like, may not be expected a priori to yield the desired norms; 
what is needed is to test a high school freshman class upon entering 
the high school or soon thereafter. One should also not fail to make 
the distinction that not all persons in grade school who made a 50 
percentile score in the distribution of extension department test 
scores) will attain that degree of success in high school; rather, if 
standardized in the manner suggested, “of those people interested 
enough to enter high school, the tests so standardized will predict 
such-and-such success, with a given correlation X.” Thus, interest, 
logically kindled through the talks given the extension students by 
the teachers, may lead the child to make the decision to go to high 
school, after which the tests will reveal with a rather high degree of 
accuracy his probable success in the high school course. A correla- 
tion of .71 between tests and high school success is undoubtedly 
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TABLE X 
ORIGINAL QUALIFICATION CARD AND FOLLOW-UP DATA OF 
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much better than a consensus judgment of many interviewers with- 
out tests would be. Proper additional selection of promising objec- 
tive tests may better this figure considerably. 

One should consider the possibility of rating pupils by their suc- 
cess in industry. This is impossible at present. The most that can 
be expected of such a vocational guidance department is to make a 
good prediction of probable high school success. 

Table X gives, for reference, the original data used in calculating 
the correlations of Table 1X involving the thirty commercial high 
school students. 


CHAPTERALY: 


GENERAL CONSIDERATIONS IN TESTING AND TRAIN- 
ING FOR PROFICIENCY AND PROMISE 
IN THE TRADES 


Some of the observations made on the trades, on tradesmen and 
on trade practices, during the assembly and standardization of 
ninety army oral trade tests may be of value to anyone who would 
compare army methods of testing trade proficiency with school 
methods. The army tests were standardized on men in industry so 
that the following remarks apply to men in industry and not to 
soldiers. 

Vocational schools may not have been “successful” in the past 
owing to the fact that they were not, in spite of their claims, aiming 
to train workers of the kind which they claimed to be training. The 
vocational schools have claimed to be training tradesmen; they 
really were attempting to train foremen. Industry has been looking 
to the trade schools for tradesmen, and necessarily has been disap- 
pointed in some cases. 

A tradesman’s training in industry is intellectually narrow in 
comparison with the training which the average vocational school 
would give him. We have already mentioned the specialization which 
is entering practically all industries and all trades. Trades are being 
split up into jobs, and jobs into operations, in order that inexperi- 
enced and unintelligent persons can quickly become competent and 
productive, and in order to obtain the profit accruing to the increased 
production made practicable by a person’s being allowed to become 
highly skilled in one simple operation. No longer does the typical 
tradesman have to do much figuring for himself. He is “sent out on 
a job,” the standard time for doing which has often been figured out 
in advance. On the job, a foreman directs his work at every step. If 
his tools get dull, he hands them in many cases to a toolgrinder; if 
his belts break, he calls the millwright whose sole business it is to 
keep the machinery running; if his work is not turning out properly, 
he may call the toolsetter; and finally, an inspector tests his products. 

In consequence of this tendency to specialization, in but few of 
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the trades are there many pertinent items about which trade test 
questions may be formed. Even in the highly skilled mechanical field 
of the toolmaker, piece-work wages are being given to some workers. 

Again, we have mentioned the use of charts, tables, and mechan- 
ical devices used in industry to eliminate arithmetical calculations 
on the part of the worker. The typical workman is disdainful of 
studying books. Along with this goes a lack of realization of just 
how dependent he really is upon the directions of others and upon 
such book-made aids to trade proficiency. As a result of this atti- 
tude, perhaps, he reads trade magazines but occasionally. These 
remarks do not apply to the foreman, who is not the typical, or jour- 
neyman, tradesman. Having but little responsibility for the job, 
once his toolchest is locked up, the average journeyman’s interest 
in trade matters largely ceases at five P. M. on week days and at 
twelve o’clock on Saturdays. | 

But few tradesmen study books on their trades after hours. 
Wherever, in army standardization of oral trade tests, a boy was 
found who was pursuing a correspondence course in his trade, he 
generally made a higher trade test score than other boys of the same 
length of experience but without the correspondence school course. 
The general consensus of opinion of foremen in such cases was to the 
effect that such boys were of more value on the job than boys who 
did not have the “ambition” to take such courses. Boys who attend 
night school seem generally superior to boys who do not, but less 
markedly so than the boys who take correspondence school courses. 
Possibly one may correctly look upon correspondence school adver- | 
tisements as an excellent sieve for picking out these boys who have 
“enough ambition to save up X dollars for self-improvement,” where- 
as the night school appeals to a lesser ability, “enough ambition to 
go to a free school for self-improvement.” 

The average journeyman, and the expert to a lesser degree, is lim- 
ited in the difficulty of trade questions which he is able to answer. It 
has been found, for instance, that “technical” questions, of the type 
which one might propound to a superintendent of a machine shop or 
to a mechanical engineer, were seldom retained for the final examina- 
tion set in the army oral questions; they were usually discarded be- 
cause there were so few passes on a question as to make it almost 
worthless in being able to differentiate between different classes of 
trade ability. As a striking example, we might cite the case of some 
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forty or fifty millwrights, journeymen and experts, to whom three 
such “difficult” questions were administered. A millwright is in charge 
of setting up machinery ina factory and keeping the belts, shafts, etc., 
in constant repair. One of the easiest of the three questions was: 
“Tf a drive pulley, 24 inches in diameter, is running at 200 revolutions 
per minute, how many revolutions per minute will an 8-inch pulley 
revolve, which is belted to it?” Only one journeyman and five experts 
were able to answer the question correctly. Many of them would not 
even attempt it. The question, it must be observed, is an oral ques- 
tion and some intelligence is needed to keep all the factors in mind 
at one time; it is not certain that the same results would be secured 
from using a blue-print of the same situation and asking the ques- 
tion with reference to the blue-print. The question was repeated in 
its entirety, as often as required by the tradesmen, and yet many of 
them seemed unable to comprehend the situation clearly enough 
even to attempt to answer it. 

Oral trade test questions must, for tradesmen in the trade, be 
limited to “practical shop kinks and everyday knowledge.” In few 
trades are there to be found as many as seventy-five questions, of 
graded difficulty as is necessary, which will fulfill all the require- 
ments of standardization questions; and of these, many will be dis- 
carded in the standardization process if rating tests are being made. 

The majority of questions must not be technical questions, and on 
the other hand must be such questions as will sample the daily habits 
of the workman. For “difficult” questions one must search for the 
type of question ordinarily regarded as “trade secrets.” These trade 
secrets, when analyzed, prove in most cases to be merely principles 
of physics, chemistry, general science, or measurement, such as any 
high school senior should know. Such, for example, is the “rule” that 
“to get the circumference of an iron ring you multiply the diameter 
by 3 1/7.” It seems at least plausible that the high repute for “skill 
and accuracy” generally conceded the general machinist, or “mechan- 
ic,” is due to a misjudgment of the public as to the difficulties in- 
volved in measuring to the one-thousandth part of an inch. Using 
the mechanical measuring equipment which is available in any 
machine shop, any boy of the intelligence level of a high school 
graduate should be able to learn to measure to that degree of accu- 
racy in about fifteen minutes. Removing one thousandth of an inch 
of stock from a piece of iron being turned in a lathe is an even 
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simpler process, with fewer chances for érror. “Expert” workman- 
ship, such as is required on but few of thé production jobs of indus- 
try, means ability to measure to the ten-thousandth part of an inch. 
For a nominal sum one may now bi sets of “size! blocks, turned 
out in quantitites, which.are “guaranteed not to vary over one hun- 
dred-thousandth part af an iaéh from the stated dimension.” Con- 
fronted by such statements in the papers, the genera! public is in- 
clined to overestimate the amount of skill, knowledge, and “trade 
secrets” possessed by the typical journeyman. 

Consistent with the overestimation of trade skill possessed by 
the typical journeyman, the general public has likewise been inclined 
to overestimate or misplace the emphasis which should be placed 
upon intelligence in the acquisition of trade proficiency. A chart, en- 
titled “Occupational Intelligence Standards” published by the Sur- 
geon General’s Office, in a pamphlet, “Army Mental Tests,” clearly 
illustrates this fact. The chart shows the range in intelligence cov- 
ered by the middle 50 per cent of each of the trades indicated. Only 
a very few workmen, such as laborers, have an average intelligence 
very decidedly below the average intelligence of the whole popula- 
tion; in perhaps three fourths of the trades of modern industry, 
“average” intelligence is quite capable of doing work in the manner 
in which it is now done; in only a very few occupations, such as the 
clerical vocations and the professions, is an unusually high average 
amount of intelligence necessary for competency in the trade. If one 
talks to a foreman who is a leader in his trade, he is rather sure to 
state the fact that an unusually high degree of intelligence is de- 
manded in order to be a satisfactory worker in is trade. The man 
prides himself on the fact that his trade is a very difficult one to 
learn, requiring years of patience and effort in order to become a very 
skilled worker. As a matter of fact, no such condition prevails in at 
least three fourths «wf the occupations available to boys entering 
industry. Such men .are prone to overrate the ability required in the 
present day to learn: the trade which they may have learned them- 
selves only by dint: of many “hard knocks.” Much of the trade 
information which fhe tradesman of thirty years ago learned only 
through experience can now be acquired by a few minutes search in 
standard reference fyooks on the trade. 

Informal tests of? ttradesmen’s ability to recall the questions asked, 
performed immed’ iately after an examination was finished, have dis- 
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closed the fact that few but-the very best and most intelligent trades- 
men can remember even, haif a dozen questions an hour later. The 
examination should proceed. with.dispatch from one question to 
another, thus giving. the subject. but little time to consider each 
question and thus fix it in his mind for later.recall. 

Questions involving: complicated mentai arithmetic are generally 
beyond the’ability of even the better tradesmen. Questions involv- 
ing the use of fractions are generally useless in oral trade tests. 
Mental computation, more elaborate than simple addition of easy 
numbers is generally too difficult for a tradesman of lower standing 
than a foreman. Questions involving many limiting conditions, or 
clauses, are unsatisfactory, owing to the inability of the tradesman 
mentally to grasp so complicated a situation. By conditions, we 
mean questions similar to the following: “(1) If you were filing brass, 
(2) on a speed lathe, (3) using a leather belt, (4) the spindle running 
at a speed of 1500 revolutions per minute, (5) what............. ra 
As a practical resuJting principle for trade test formulation, a ques- 
tion should not include many conditions if it is to be satisfactory. 
Essential elements of knowledge may generally be found in such 
questions, and the question may then be split up into a number of 
simpler questions. For the same reason, those questions which con- 
tain but few words are the surer of being correctly comprehended by 
the workman. If possible, oral questions should be limited to less 
than twenty-five words in length, and the shorter the better. Short 
questions have the additional value of being quickly administered. 
If one were to use twenty questions of twenty-five words each, or 
forty questions of ten words each in approximately the same exam- 
ination time the latter procedure is much more likely to give a selec- 
tion of tradesmen in the correct order of their abilities. Language 
ability seems more necessary for a foreman than for an average 
journeyman; consequently, the longer, more involved questions will 
be a better measure of foremanship than journeymanship ability. 
The general education possessed by tradesmen is such as to make one 
doubtful as to whether success is due to such education, or to intel- 
ligence and other qualities of character. 


THE EDUCATION OF TRADESMEN 


The belief has been fairly common that the tradesman is an 
“educated” man, and the laborer an “uneducated” man. A tabula- 
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tion, by trades, of the number of school grades completed by 924 
tradesmen distributed among thirty trades in the city of Pittsburgh 
shows that: 

1. The median grade at which the various groups of tradesmen 
left school does not vary greatly, the range being from 7.0 in the case 
of horseshoers to 10.0 (second year High School) in the case of car- 
buretor man. 

2. In the case of six trades, samplings of tradesmen from Pitts- 
burgh and Newark (N. J.) the median grades of the men in the same 
trades varied in the two cities less than an average of .3 grade. 

3. The trades were given a trade rank order depending upon the 
median grade of education of their tradesmen. The correlation, p = 
.79, between the grades and the average trade rank per grade, indi- 
cates that, in general, poorly educated men tend to gravitate into 
trades of which poor education is the more typical, and vice versa. 

4. Of all the tradesmen, only 26.4 per cent had persisted through 
a part of high school or more; and 2.8 per cent had completed a 
part of a college course or more. 

5. Experts are generally the best educated of the three classes of 
tradesmen, apprentices next best, and journeymen least well edu- 
cated. This seems to indicate that the journeyman stage is a sifting- 
out stage, possibly a stage of adaptation of mentality and mechanical 
aptitude or interest to the job, during which time the better educated 
(highly intelligent) men become experts, foremen, owners, and the 
like, leaving behind a group of less capable men, “doomed to be jour- 
neymen.” 

6. White laborers, from the Camp Dix personnel office files, were 
found to be less well educated than colored laborers, while both were 
very inferior in education to tradesmen. 

7. General clerks in New York City were found to have a higher 
median education than any of the trades. 

The educational chart on page 104 (Fig. 29) shows comparable 
results for the above trades and occupations in regard to educa- 
tion. The results here presented were compiled as the result of 
interviews with tradesmen in the shop. The length of the bars of this 
chart indicates the range of the middle 80 per cent in regard to educa- 
tion. The cross-hatched area of the bar indicates the range of the 
middle 50 per cent. These data are based on a rather small number 
of cases, the numbers upon which each distribution is based being 
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indicated on the left of the chart. It will be noticed that such trades 
of this list as are included in the army intelligence chart parallel re- 
markably the intelligence of those groups. A correlation between 
the median ranking in education and intelligence of fourteen trades, 
for which data are available gives p = .78 = .10. 


TABLE XI 


PERCENTAGE DISTRIBUTION OF MEN, ACCORDING TO NUMBER OF 
GRADES OF SCHOOL COMPLETED. GENERAL SUMMARY OF 
174,857 MEN, VARIOUSLY CLASSIFIED 
SS EEE EE ee ee ee 


Per Cent Retained in School 


Per Cent Leaving School 


Grade to Enter Succeeding Grade: 
A B G D ES A B Cc D E 

None 262 1.0 4.7 0.3 97.8 tM 99.0 95-3 99.7 
Xr 0.7 0.3 0.5 6 97.1 San 98.7 94.8 99.7 
2 be 4.8 0.8 0.9 8 95.6 aA 97.9 93-9 99-7 
3 2.8 1.2 1.8 0.2 92.8 te 96.7 92.1 99.5 
4 5.0 2.8 3-6 0.5 87.8 95.2 93.9 88.5 99.0 
5 8.0 4.0 4-7 3-4 1.9 79.8 QOI.2 |* 89.2 85.1 97.1 
6 11.6 13.7 11.8 7.6 2.9 68.2 78.5 77.4 7758 94.2 
7 10.6 22.3 9.8 10.3 10.0 57.6 56.2 67.6 C72 84.2 
8 31-5 35.0 45.1 34.6 48.3 26.1 2562 22.5 32.6 35-9 
1H. S. 9.7 Tis 6.1 6.0 6.9 16.4 9.9 16.4 26.6 29.0 
It H.S. 6.2 6.6 6.4 6.1 8.9 10.2 33 10.0 20.5 20.1 
III H. S. 2.9 2.2 2.7 4.2 BL 7.3 5 Aa 73 16.3 15.0 
IV H.'s: 4.7 er ¢ 5.0 Gf 10.3 2.6 2 2.3 9.2 4.7 
A College 0.9 2 0.5 7.2 1.7 2 1.8 wie @ S85 
B College 0.6 2 0.3 ae 3% 2 25 erie 2.4 
C College 0.3 2 0.4 9.2 0.6 0.8 2 E.£ a 1.8 
D College 0.7 2 Tok L.7 0.1 2 0.0 EN 0.1 
Post-Grad. 0.1 2 0.0 0. 0.0 0.0 0.0 0.0 0.0 

Total 100.0 100.0 100.0 100.0 100.0 Per Cent with Part H. S. or Better: 


26.1 21.2 2255 32:6 35.0 


No.of Men 18,266 120,289 924 30,583! 4,795 Per Cent with Pari College or Better: 
Av. Grade 7.43} 7.53 7.83 8.09 8.86 2.6 2 as 9.2 4-7 


° 

1 Post-graduate college counted as seventeenth grade. 

2 College not tabulated. Hence all percentages slightly too high. 

3 Only 43,358 cards selected at random from the 120,289 were tabulated. 

4 Excludes “aliens, both friendly and enemy.” 

5 Samplings only of three groups taken: (a) Non-referred men, (b) referred and 
placed, (¢) referred and not placed. Weighted for column E according to frequency of 
occurrence:E = (a X17) + (6 X 7) + (¢ X 5). 

6 Application card did not provide for entry of second and third grades. 
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TABLE XI. Key To CoLtumMns 


A. Unemployed men at Columbus and Dayton, Ohio, Free Employment Offices 
1916-1917 (Toops, H. A. and Pintner, R., “Mentality in Its Relation 
to Elimination from School,” School and Society, Vol. 7, No. 173 and No. 174, 


1918, pp. 507-510 and 534-539). 

B. All Working Boys 16, 17 and 18 years of Age subject to Military Duty, resid- 
ing in villages or towns of 5000 population or over, 1919. Survey of the 
Military Training Commission, State of New York. Data kindly furnished 
by Mr. H. G. Burdge, Director of Vocational Training Bureau. 

C. Tradesmen, all degrees of skill distributed among 31 trades, Pittsburgh, Pa. 
Examined in War Department standardization of Army Oral Trade Tests. 
(Toops, H. A. and Pintner, R., “Educational Differences among Tradesmen,” 
Journal of Applied Psychology, Vol. 3, No. 2, 1919, pp. 33-49.) 

D. Minneapolis Draftees. “Aliens excluded.” Malone, T. J., “When Boys Leave 
School,” American Review of Reviews, Vol. 60, No. 6, Dec. 1919, pp. 627- 
630.) 

E. Returned Soldiers, applying for work at the Reemployment Bureau of New 
York City, 1919. (Toops, H. A., Unsigned news note in the World, New 
York, Sept. 21, 1919.) A somewhat selected group, containing many clerical 
workers. 


Table XI is a distribution table, compiled from all data available 
to the author, of the grades at which various groups of working men 
left school. It shows that tradesmen are not, in general, either a 
superior or an inferior group of people so far as education is con- 
cerned. This table serves to emphasize the fact that the typical 
tradesman is not a high school man. Many of our vocational schools 
attempt work of high school grade. Where such is done, it obviously 
will fail to affect the lives of the three fourths of working people who 
have left school before formal vocational instruction is begun. 

We are interested also in the relationship of the amount of educa- 
tion possessed, as measured by the grade at leaving school, to the 
degree of proficiency in the trade. 


THE RELATIONSHIP OF GENERAL EDUCATION TO PROFICIENCY 
IN A TRADE 


Many articles have been written, and much discussion has been 
devoted to the problem, “Does education pay?” Groups of educa- 
tors, intent on proving the worth of some specialty in education, or of 
education in general, have used the argument that such education 
pays because it may be statistically demonstrated that people with 
such education receive more wages, rise to higher positions of influ- 
ence, and the like, than their unlearned brothers. That these argu- 
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Fic. 29. DISTRIBUTIONS OF GRADE AT LEAVING SCHOOL OF OCCUPATIONAL 
Groups. 


The shaded area of each bar shows the range of the middle 50 per cent; the 
whole bar shows the range of the middle 80 per cent. The vertical line indicates 
the median of each trade. 
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ments may be fallacious is evident if one but considers the possibility 
that the group in question may have been a selected group who 
would have succeeded equally well in industry without the educa- 
tion. 

The author was granted permission to evaluate the qualification 
cards of one hundred and thirty soldiers, just returned from 
overseas, and applying for jobs as general clerks. Using as a cri- 
terion of occupational worth whether or not a man was hired when 
referred to a prospective employer for a job, and evaluating against 
this, by partial correlation method, the talents for the job, available 
from the qualification card, we obtain the following equation: 


i =~ 1 Sn ~ (a ts (8 
C1G JT x4 100( ee Xe Ae 25 


al a) 


in which, x; is the predicted job value (criterion) 
(not hired = score of I 
hired = score of 2) 


x2 is the age in years. 

x3 is the grade at leaving school. 

x4 is the marital condition, (single = score of 1; 
married = score of 2) 

xs is the number of dependents. 


The composite score, compiled by use of the equation, yields a 
correlation of .405 with the criterion. The fact that, other things 
being equal in this composite of talents, the more education a pros- 
pective employee has the less desirable was he considered by the 
prospective employer, may be merely a reflection of the economic 
situation at the time (spring of 1918). Again, one may say that the 
criterion is too rough for accurate results. The criterion is admittedly 
rough and defective. The important consideration is that grade 
at leaving school receives a negative weighting in the composite of 
talents for the job. 

The author was also privileged to secure intensive data on a group 
of nineteen eyelet machine tenders in a large brass factory. This job 
is semi-automatic in that the tender merely has to keep a number of 
machines fed with strip metal, and to watch for defects in the prod- 
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uct, and detect imperfect working of the machine. The following 
measures of job worth were obtained: . 


Variable 1. (Criterion). Average daily piece-rate wage over a period of two 
weeks, reduced to a comparable 10-hour day basis, exclusive of overtime and 
contingent wage. 

Variable 2. Average of three rankings in trade ability of the workers made by the 
foreman, one week apart, using the slip arrangement order of merit method. 

Variable 3. Length of experience on the job to the nearest .o1 part of a year. 

Variable 4. Oral trade test score. One-word-answer form of question, adminis- 
tered orally and individually by the author. 

Variable 5. Grade completed in school. 


Using wages as our criterion, it is possible to determine the money 
value contributed to daily wage by one question answered on a 
trade test, by one grade in school completed, etc., all other talents for 
the job of the composite being constant. 


The formulae resulting are: 


In terms of deviations, 


(1) x 


ort 


=43305.— + 0890 5204 re 
a) o3 v4 tome 


) 


Or, in terms of gross measures, 
(12) X, = .0508X_ + .1727X3 + .0456X4 — .1408X;5 + K. 


The correlation of the composite with wages is .687. Again we 
notice that, although it is an advantage in wages received for a work- 
man to be rated highly by the foreman, to have long experience at 
the trade, and to receive a high trade test score, grade at leaving 
school is a disadvantage, when other talents are equal, to the extent of 
fourteen and a fraction cents per day per grade of school attained, in 
this composite; that is, grade at school receives a negative weight- 
ing in spite of its positive correlation with the criterion. When our 
educators are willing to evaluate their products by the same methods 
which they would have industry use in evaluating theirs, both will 
profit by the renewed interest in education. One should not general- 
ize from two isolated instances such as these. The roughness and 
defectiveness of any criterion by means of which to evaluate the 
worth of any educational program, any selective method, makes 
this form of analysis difficult. The method of analysis suggests prob- 
lems rather than solves them. 
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It is interesting to note, in the first of the above equations, that 
“trade test ability” is the most important contributor to wages, fore- 
man’s judgments of his men next best, while mere experience in 
years is least important of all four, as judged by the magnitude of the 
partial correlations. 

School surveys, designed to test the needs for vocational schools, 
have always been made with the point of view that such education 
would pay. Hence such surveys have literally been searches for facts 
to “prove the point” that vocational education would pay. This has 
resulted oftentimes in misplaced emphasis, to the detriment of voca- 
tional education. The problem should be “what kind of vocational 
education will pay?” rather than a search for facts to justify voca- 
tional education in general. 

In such surveys, job analyses are commonly made of a number of 
trades. The personal requirements, personnel specifications, of 
workers in the trades (for vocational education has commonly meant 
trade education) are commonly set forth. These have often been set 
down, possibly on the basis of a few interviews with foremen, or 
worse still, on the basis of the opinion of some one or two authorities 
in vocational education as to what ought to be the personal require- 
ments of the worker. 

As contrasted with such points of view, we have that developed in 
the trade-test work of the army, in which the requirements of the 
worker are any requirements above the abilities of the x-percent- 
ile man, his standing in the trade being derived independently of the 
talents, or tests, for the job,—preferably being some comparative 
measure of production. 

When requirements for the job are viewed in this light, and sub- 
jected to statistical treatment, it will probably appear that many of 
the traits now considered important will prove to have low partial 
correlations with demonstrated ability on the job. 

It is instructive to think of the total sum of any industrial popu- 
lation’s talents, at any one given time, as being a definitely limited 
quantity of talent; thus 1,000 men together will have had 8,000 
years of school attendance, and so on. When education is considered 
in this light, almost all large groups of workmen will be found to have 
a very large percentage of “successful” men who fall far short of the 
educational standards set up by ordinary job analyses. Table XI 
gives a summary of the distribution of grade at leaving school of 
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174,857 workmen, as tabulated from the men’s own statements 
recorded on questionnaires. A casual inspection of this table will 
prove that about half of our working population, with less than 
eighth grade education, find jobs somewhere. When our vocational 
educators find vocational education for this half of our population, 
then such education will more nearly meet the needs of those not 
“predestined” to success even without the education. 

As a partial solution of the problem of vocational placement, we 
need to recognize traits as being of variable worth, between definite 
limits. The present tendency is to set up merely a minimum limit 
(usually an “ideal” limit) which, because of a definite limitation of 
the total sum of human talents, it is ordinarily impossible to main- 
tain. We should be willing to admit that possibly a sixth grade edu- 
cation is a minimum limit for general clerks, that possibly not more 
than one year of college education is desirable, and that within the 
aforementioned minimum’ and maximum limits, a seventh grade 
education fits better for the job than a sixth’grade, an eighth than a 
seventh, and so on. 

This view is at total variance with the economic philosophy which 
would attempt to educate only the few in order that the world might 
be well supplied with “workers.” Instead, it means conservation of 
our human resources by reserving to the unintelligent those jobs, 
any harder than which they are incapable of performing with accept- 
able speed and accuracy, in short, the highest type of vocational 
guidance philosophy. 


THE USE OF TRADE TESTS AS INCENTIVES TO LEARNING 


It is even more difficult for the student of trade subjects to know 
when he has done his work well than for the academic or scientific 
student. The academic student, with whom learning is largely con- 
tent learning, may usually easily classify categorically the amount 
and quality of his lesson learned as “the amount learned to the 
point of recall.” His test of knowledge and skill gained is the simple 
test of recall. The scientific student may easily apply the same test 
to his content matter; the student working on experimental work 
usually has the self-applicable test of whether his figures agree with 
the expected figures of the textbook, whether his chemical analysis 
reveals the same chemical elements which the chemistry instructor 
put into the test solution, and so on. Trade instruction usually con- 
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tains a minimum amount of content instruction, and much manual 
manipulation. It is difficult to measure the product of manual ma- 
nipulation subjectively. 

The use of self-administrative performance trade tests for meas- 
uring mechanical products may be made of great use as incentives to 
better trade performance in the case of students sufficiently advanced 
to apply the tests. Substantially this plan is already in operation in 
two engineering colleges known to the writer. 

In the first of these schools, a simple time standard of performance 
quality being judged “passable” by the student foreman, has been 
set up by the best average records of former classes. Those students 
who beat this time standard are allowed extra credit in grades for the 
superior performance. There is great rivalry among the boys to 
turn out product as rapidly as possible, and a keen group coéperative 
spirit of determination to beat the best record of the previous classes. 
The method is lacking in the important feature of failing to suffi- 
ciently stress “good” work as well as quantity of work, now stressed. 

In the second school, percentile norms of performance have been 
set up. This has obvious advantages over a categorical “standard 
time,” as the student can see for himself not only that he is better or 
poorer than the standard, but just how much better or poorer. For 
school purposes the merit of a timed production of work obviously 
depends upon a measure of dispersion of the distribution of times of 
former students as well as upon the absolute values of the respective 
times taken. In industry, the boy who turns out a product in half 
the time of another is worth twice as much wages as the latter. In 
vocational school work, the primary emphasis should be upon the 
acquirement of a certain amount of trade skill, the time required 
being a matter of secondary importance. 

If percentile norms (or index numbers derived from an assump- 
tion of a normal curve of distribution of trade ability measured in 
o’s) were set up for time of performance of all vocational school 
projects, and if a similar, or preferably an absolute standard of 
quality of performance were likewise set up for all vocational school 
projects, then every task performed by the vocational student would 
be a cumulative test of the degree of his skill at the trade. These 
tests should be self-administrative. In this simple manner the 
student may be continually faced by the two generalized measures 
or objectives of trade work, quality and quantity of product. 
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One of the big advantages of such tests of daily production is the 
psychological advantage of interest in the work which is created by 
receiving the rating immediately. A keen interest is usually kindled 
by any formal examination. If the ratings are not received for sev- 
eral days or weeks, during the time required by the teacher for scor- 
ing the papers, much of the original interest in the ratings has been 
lost by the time the ratings are received. The value of a series of 
daily grades, determined by the student himself, in keeping him 
interested in his work can be illustrated by a few percentile ratings 
which presumably might be received: 75, 83, 70, 86, 83, 20. Were 
the student required to plot his grades as a daily, or job require- 
ment, the graph afforded by the above percentile ratings would be 
sufficient incentive to most boys to redouble their efforts on the 
seventh day. And, inasmuch as the percentiles are based on the 
work of past classes, the poor pupil has, in effect, an absolute stand- 
ard to beat and not the “unbeatable” record of relative standing in 
his class. “Working for grades” under such’conditions of objective 
measurement, free from the instructor’s bias, is a highly laudable 
performance. : 

For rating quality of product, a series of “limit” or “snap” gages 
can be easily provided for many products. Where judgments of 
quality are needed, uniform blanks of analyzed “traits” similar to 
the army performance test scoring points might be provided and the 
student be required as part of his task to rate his own product, being 
graded on his ability to rate his product as well as on the excellence 
of the product itself. 

Formal tests of performance on analyzed important aspects of 
trade proficiency may also be made self-administrative and self- 
scorable. The performance measurement test of Chapter I, designed 
to test the student’s ability to use measuring instruments, is scorable 
by the use of a stencil in but a minute of the student’s time. It would 
seem possible to devise such diagnostic tests of individual weak- 
nesses and strengths in a large number of the operations and pro- 
cesses found in the machine or woodworking shop. 

The self-scorable feature of the one-word-answer form of test has 
been found to have a big appeal to the interest of students, in classes 
in content subjects, where the method has been given a trial. 
Examinations given weekly or oftener, taking only a half hour or so 
for the complete examination including the scoring, may be found 
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to be of much more value, both as incentive and as rating method, 
than much longer examinations given at longer intervals and scor- 
able only by the teacher at a great expense of time and effort. Using 
such tests, it is not even necessary that the questions be mimeo- 
graphed. The teacher, standing before the class, merely reads the 
questions and waits a short time for the students to record their 
answers before proceeding to the next question. All students may 
then exchange papers, the answers being scored either right or wrong 
by the pupils as the teacher slowly reads the correct answer list. The 
results thus far show that this method may be applied successfully 
with students as low in ability as the third semester of high school. 
The method will be found applicable, in some subjects, to the upper 
grades of the elementary school as well. 


APPENDIX 


STATISTICAL METHODS 


The Pearson correlations herein reported, of all variables of 
classes with n classes insofar as it was practicable to plot them, were 
computed by means of stencil and tables of deviation squares by fre- 
quencies, using the following specific formula: 


Yl cox: + 2Y?) — 2X - v)} — (2X) (ZY) 
(3) re 
VN (2X?) — (2X)? VN (ZY?) — (ZY)? 


in which X and Y are stencil steps. 

For the fourfold Pearson correlations entering into partial correla- 
tion regression computations, such cases as the correlation between 
single—married and not-hired—hired, the formula used is, 


(14) v= ad — be 
Via +c) (6+ 4) @+b) (+4) 


in which ad represents the product of the (single—not-hired) by the 
(married—hired) frequencies. 

For the Pearson correlations between a variable of 1 classes and a 
categorical variable of two classes, the following formula was used: 


_ __N&(b-Y) — B (BY) 


(15) ee 
VA-B VN(SY2) — (ZY)? 


, in which, 


A is the total frequency of the lesser degree of the cate- 
gories of the X-variable, 
B is the total frequency of the greater degree of the cate- 
gories of the X-variable, 
~b'Y is the sum of the b (greater degree categorical frequen- 
cies) by their respective Y-gross scores. 
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For intercorrelations of n variables, using a calculating machine, 
it has been found very advantageous: 


(1) To arrange the crude scores in parallel columns, Xj, X2,...Xn, 

(2) subtract from each gross score the lowest gross score of the col- 
umn yielding the remainders, X1 — c, X2 — €2,....Xn — Cny 
and 

(3) then compute the correlation in parallel columns headed, 


Xs Gi) 5m (aa) as (ig, 2g)? Xa 01) (Xe =), 
(M ze C1): (Xs Dae €3) je (Mp ae Cnt) (Xn | Cn). 
Most of these multiplications may be done mentally (the squares of 
all numbers from I to 40 may be readily memorized) or by means of 


a simple multiplication table. The method will be found to be par- 
ticularly efficient when JN is not over 50 persons. The formula used is. 


NZXY — ZX-LTV 


(16) C—O 
V NSX? — (3X)? VNZY? — (SY)? 


The general arbitrary scoring formula for tests given by the mul- 
tiple choice method, and applicable either to work-limit or time- 
limit methods of scoring, where n is the number of choices, R the 
number of attempted questions answered correctly, and W is the 
number of attempted questions answered wrongly (or omitted), 
and A is the number of questions attempted, is, 


meas ea ia 


(17) oe pes 
The probable error of an individual score is given by the 
formula, 
(7 ) R-W 
(18) P. E., = .6745 eI aa a 
If a criterion is available, so as to use the partial regression equa- 
tion of 


(19) S=R-+C- W, wherein C has the value determined by 
Thurstone,! 


1 Thurstone, L. L., “A Scoring Method for Mental Tests.” Psych. Bul., Vol. 16, 
No. 7, 1919, DD. 235-240. 
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(20) opeeaeds: ("ir - tRw_~ tw) 
ow ("iw 'Rw — TR) 


Then the generalized formula for the probable error of an indi- 


vidual score is, 
| RW 
A leis 
The formula for determining the reliability of 1 forms of a test 
with 7 other forms of a test the reliability of one form with a second 
being 71, is, 


(21) PLE. = 6745 |\(C-—"1) 


N+ Ty 
I +- (n = 1) Ti 


This equation may be solved for n, yielding, 


(22) Tran = 


Yan (1 a ri) 4 


(23) n= 
Yu (1 a Ca) 
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Fic. 30. CHART FOR FINDING PROBABLE ERRORS OF PEARSON 7’s. 
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PROBABLE ERRORS OF CORRELATION COEFFICIENTS 


For the convenience of those who may be interested in the P. E.’s 
of the correlation coefficients herein given, the chart of Fig. 30 is 
published. Such straight line charts may be adapted to solving 
many equations commonly used in routine test work. 

To use the chart: Locate the 7, of which the P. E. is desired, along 
the lower base line. Project this vertically by an imaginary line 
until intersecting the diagonal line which represents the number of 
cases on which the correlation coefficent is based. Then, project 
this intersection either to the right or left hand margin of the chart 
where the P. E., may be read directly. Interpolation must be done 
for values of N not on the chart, and for values of 7 not on the .05- 
point division lines. Even with the interpolation, the accuracy is as 
great or greater than is commonly required. 
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